-- Logs begin at Fri 2022-04-29 17:51:13 UTC, end at Fri 2022-04-29 19:09:50 UTC. -- Apr 29 18:28:33.470368 dv-cinder-839987-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Apr 29 18:28:33.479379 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Apr 29 18:28:33 2022] *** Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: nodename: dv-cinder-839987-1-iscsi Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: machine: x86_64 Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: clock source: unix Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: pcre jit disabled Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: detected number of CPU cores: 4 Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: current working directory: / Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: detected binary path: /usr/bin/uwsgi-core Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: your processes number limit is 47680 Apr 29 18:28:33.481437 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: your memory page size is 4096 bytes Apr 29 18:28:33.482059 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: detected max file descriptor number: 2048 Apr 29 18:28:33.482059 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: lock engine: pthread robust mutexes Apr 29 18:28:33.482059 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: thunder lock: enabled Apr 29 18:28:33.482059 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Apr 29 18:28:33.482059 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Apr 29 18:28:33.527023 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: Python main interpreter initialized at 0x5644d41e29c0 Apr 29 18:28:33.527023 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: python threads support enabled Apr 29 18:28:33.527023 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: your server socket listen backlog is limited to 100 connections Apr 29 18:28:33.527023 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: your mercy for graceful operations on workers is 90 seconds Apr 29 18:28:33.527402 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: mapped 403077 bytes (393 KB) for 2 cores Apr 29 18:28:33.527479 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: *** Operational MODE: preforking *** Apr 29 18:28:33.527754 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: *** uWSGI is running in multiple interpreter mode *** Apr 29 18:28:33.527754 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: spawned uWSGI master process (pid: 80117) Apr 29 18:28:33.527983 dv-cinder-839987-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Apr 29 18:28:33.532525 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: spawned uWSGI worker 1 (pid: 80131, cores: 1) Apr 29 18:28:33.532525 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: spawned uWSGI worker 2 (pid: 80132, cores: 1) Apr 29 18:28:33.532525 dv-cinder-839987-1-iscsi devstack@keystone.service[80117]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Apr 29 18:28:39.212242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Apr 29 18:28:39.212242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: from cryptography.utils import int_from_bytes Apr 29 18:28:39.213092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Apr 29 18:28:39.213092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: from cryptography.utils import int_from_bytes Apr 29 18:28:39.213580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Apr 29 18:28:39.213580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: from cryptography.utils import int_from_bytes Apr 29 18:28:39.213687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Apr 29 18:28:39.213687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: from cryptography.utils import int_from_bytes Apr 29 18:28:39.262262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Apr 29 18:28:39.262432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Apr 29 18:28:39.262520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Apr 29 18:28:39.262600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Apr 29 18:28:39.262660 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] command line args: [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Apr 29 18:28:39.262726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] command line args: [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Apr 29 18:28:39.262970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Apr 29 18:28:39.263053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Apr 29 18:28:39.263120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ================================================================================ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Apr 29 18:28:39.263179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ================================================================================ {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Apr 29 18:28:39.263242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] admin_token = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.263309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] admin_token = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.263488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_dir = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.263573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] config_dir = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.263634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.263694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.263860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_source = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.263947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] config_source = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.264008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] debug = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.264070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] debug = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.264144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.266610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.266610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.266768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.266828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.266891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.266960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_config_append = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_config_append = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_dir = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_dir = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_file = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.267995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268314 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.268995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269510 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] publish_errors = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.269950 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] publish_errors = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.270697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.270911 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.270911 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.271258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.271450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.271636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.271822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.272099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.272292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.272484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.272724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] standard_threads = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.272891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] standard_threads = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.273114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.273287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.273480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.273674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.273861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.274046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.274274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_journal = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.274439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] use_journal = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.274633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_json = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.274825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] use_json = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.275017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_stderr = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.275211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] use_stderr = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.275406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_syslog = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.275779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] use_syslog = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.275954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.276105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Apr 29 18:28:39.276241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.276552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.276707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.276843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.277011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.277169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.277300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.277432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.277570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.277697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.277822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.277953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.278085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.278296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.278426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.278551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.278676 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.278811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.278937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.279070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.279197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.279324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.279324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.279561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.279561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.279813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.279951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.280083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.280210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.280335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.280468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.280635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.280771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.280904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.281037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.281164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.281289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.281423 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.281549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.281690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.281823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.281948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.282073 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.282209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.282335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.282461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.282597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.282732 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.282859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.282992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.283119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.283244 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.283369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.283565 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.283698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.283836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.283963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.284090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.284222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.284362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.284519 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.284728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.284864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.284990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.285115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.285240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.285381 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.285516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.285649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.285777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.285902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.286026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.286151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.286291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.286418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.286542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.286669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.286794 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.286927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.287053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.287188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.287314 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.287445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.287571 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.287705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.287831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.287956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.288087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.288212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.288337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.288473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.288640 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.288770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.289982 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.290149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.290463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.290668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.290836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.291026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.291211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.291703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.291943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.292203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.292366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.292692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.292891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.293053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.293251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.293474 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.293636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.293822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.294008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.294200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.294386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.294611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.294773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.294959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.295153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.295312 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.295441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.295622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.295759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.295917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.296055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.296185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.296319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.296455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.296638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.296779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.296908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.297035 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.297162 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.297288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.297425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.297560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.297688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.297815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.297971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.298101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.298228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.298362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.298490 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.298616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.298742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.298867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.298993 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.299131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.299270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.299398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.299524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.299650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.299783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.299783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.300039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.300165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.300292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.300418 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.300580 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.300580 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.300937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.300937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.301186 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.301312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.301448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.301576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.301702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.301827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.301952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.301952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.302198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.302325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.302451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.302578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.302713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.302863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.302999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.303126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.303251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.303377 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.303503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.303629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.303762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.303898 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.304037 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.304204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.304335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.304468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.304656 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.304796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.304933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.305061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.305187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.305325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.305452 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.305578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.305712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.305840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.305966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.306091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.306216 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.306348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.306483 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.306625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.306751 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.306876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.307071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.307201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.307341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.307470 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.external = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.307597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.external = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.307724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.307849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.307975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.308133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.308273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.308420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.308583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.password = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.308715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.password = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.308852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.token = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.308987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.token = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.309121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.309257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.309385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.309511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.309637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.309765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.309901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.310029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.310155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.310292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.310420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.310553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.310689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.310817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.310943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.311076 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.311204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.311333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.311469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.311597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.311725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.311853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.311989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.312115 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.312250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.312378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.312537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.312670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.312670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.312911 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.313048 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.313176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.313176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.313427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.313556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.313682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.313682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.313949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.314077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.314275 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.314418 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.314561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.314700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.314828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.314956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.315083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.315210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.315337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.315474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.315613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.315748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.315874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.316002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.316129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.316332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.316462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.316614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.316755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.316885 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.317011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.317148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.317276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.317403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.317550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.317679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.317806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.317940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.318069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.318196 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.318338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.318537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.318703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.318843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.318971 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.319098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.319227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.319364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.319492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.319627 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.319755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.319881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.320007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.320133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.320260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.320394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.320570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.320703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.320840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.320968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.321094 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.321238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.321367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.321493 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.321493 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.321731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.321866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.321995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.322121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.322246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.322374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.322499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.322733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.322873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.323012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.323141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.323276 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.323404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.323540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.323668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.323797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.323924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.324050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.324177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.324312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.324441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.324593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.324722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.324907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.325050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.325186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.325314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.325442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.325571 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.325697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.325823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.325957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.326085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.326218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.326345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.326471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.326653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.326789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.326917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.327043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.327169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.327295 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.327422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.327557 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.327685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.327812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.327939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.328064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.328190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.328327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.328550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.328698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.328828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.328955 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.329082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.329220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.329348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.329475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.329601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.329729 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.329861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.330003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.330132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.330258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.330396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.330531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.330665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.330802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.330930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.331056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.331183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.331309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.331436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.331572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.331700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.331827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.331953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.332078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.332217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.332353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.332586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.332753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.332887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.333014 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.333139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.333277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.333405 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.333532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.333658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.333784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.333913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.334059 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.334188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.334315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.334447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.334576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.334711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.334838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.334966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.335092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.335218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.335344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.335479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.335606 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.335733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.335869 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.335998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.336124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.336263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.336391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.336590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.336754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.336885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.337013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.337150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.337284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.337411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.337537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.337673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.337800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.337934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.338064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.338237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.338369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.338496 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.338623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.338757 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.338886 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.339013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.339141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.339268 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.339394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.339539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.339668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.339668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.339908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.340041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.340169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.340305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.340433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.340582 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.340719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.340848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.340974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.341108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.341238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.341375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.341502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.341628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.341754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.341954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.342084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.342212 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.342339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.342465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.342591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.342725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.342854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.342980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.343112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.343251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.343379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.343514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.343642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.343768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.343895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.344021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.344536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.344684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.344813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.344940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] role.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.345067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.345193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] role.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.345330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.345467 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.345595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.345722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.345848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.345974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.346102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.346236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.346364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.346492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.346618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.346745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.346872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.347007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.347145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.347272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.347398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.347526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.347652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.347787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.347914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.348041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.348167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.348294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.348429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.348579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.348708 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.348845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.348972 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.348972 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.349223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.349352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.349478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.349478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.349718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.349844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.349978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.350106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.350232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.350358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.350494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.350621 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.350768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.350897 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.351024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.351151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.351277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.351404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.351538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.351667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.351794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.351921 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.352046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.352187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.352323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.352451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.352600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.352729 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.352855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.352984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.353121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.353251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.353379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.353505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.353631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.353788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.354023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.354169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.354298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.354426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.354563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.354752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.354935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.355078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.355212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.355340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.355468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.355603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.355732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.355860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.355999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.356127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.356254 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.356388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.356539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.356682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.356816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.356950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.357084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.357229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.357364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.357497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.357630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.357763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.357908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.358051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.358186 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.358319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.358453 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.358587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.358739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.358739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.359051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.359186 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.359320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.connection = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.359463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.359612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.359612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.359880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.360028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.360176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.360311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.360445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.360651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.360804 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.360939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.361084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.361222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.361356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.361490 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.361623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.361757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.361910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.362046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.362189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.362325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.362465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.362600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.362743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.362879 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.363012 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.363147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.363281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.363416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.363627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.363775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.363910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.364052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.364191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.364327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.364476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.364751 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Apr 29 18:28:39.364888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.364888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Apr 29 18:28:39.364888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Apr 29 18:28:39.375027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.375245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.375390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.375540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.375678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.375818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.384154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.384313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.412609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.412851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.412997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.413151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.465355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.465745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.516471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.516802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.516977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.517233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.581769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.581955 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.582191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.582407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.582556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.582649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.582814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.582940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.583279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.583594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.583956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.584268 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.606803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.607045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.607156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.607156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.607390 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.607516 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.607611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.607792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.607974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.608071 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.608380 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.608718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.609026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.609336 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.620418 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.620626 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.634555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.634737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.634965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.635055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.635285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.635357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.635607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.635702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.635768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.635830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.635905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.636017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.636386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.636386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Apr 29 18:28:39.653085 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.653305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Apr 29 18:28:39.653540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.653681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.653753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Apr 29 18:28:39.653816 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Apr 29 18:28:39.654092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.654177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.654242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Apr 29 18:28:39.654325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.654568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.654649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.654713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.654773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.655013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.655201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.665411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.665671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.665933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.666052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.666268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.666444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.666628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.666927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.674589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.675278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.677887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.678170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.678394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.678554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.678654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.678832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.678913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.679421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.679514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.679579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.679652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.679717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.679778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.679839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.679900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.680302 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.680444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.680619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.680759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.680897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.680897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.681182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.706906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.707859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.708506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.708775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.709492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.709774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.714048 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.714289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.714708 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.715045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.715342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.715788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.722330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.722573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.723384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.723673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.727035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.727323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.727569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.727865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.728083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.728237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.728382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.728580 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.728726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.729034 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.729320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.729529 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.729529 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.729529 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.730258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.730258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.747967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.748198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.749046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.749330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.752796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.753874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.756390 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.756681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.757547 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.757784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.761969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.763040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.765704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.766687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.769364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.769658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.769905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.770203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.770406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.770650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.770798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.770949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.770949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.771217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.771518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.771794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.789768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.790012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.790300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.790549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.790864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.791188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.791468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.791685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.798472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.799544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.800149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.800384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.801284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.801534 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.803571 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.803821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.804117 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.804366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.804696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.804900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.805067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.805210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.805508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.805789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.818130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.818350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.818657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.818908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.819088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.819331 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.819494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.819637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.819775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.819911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.820058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.820197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.820563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.820784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.837856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.838994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.841151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.842280 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.844787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.845945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.846583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.847664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.848128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.848376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.848691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.849335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.849590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.849896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.855995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.857339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.857705 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.858928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.860909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.861192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.861575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.861824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.862164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.862423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.862831 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.863106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.871687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.871971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.872189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.872454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.872771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.873007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.880069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.880294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.880625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.880887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.880964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Apr 29 18:28:39.881148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.881220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.881433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.881703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.881970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.882491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.882780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.883058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.883328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.883396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.883596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.883724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.883865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.883934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.884137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.884207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.884412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.884508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.884700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.884771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.884970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.885446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.885908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Apr 29 18:28:39.922433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5644d41e29c0 pid: 80132 (default app) Apr 29 18:28:39.924243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5644d41e29c0 pid: 80131 (default app) Apr 29 18:28:39.925892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc165f49-11dc-41f7-a164-dc9d4b000594 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:39.926185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc165f49-11dc-41f7-a164-dc9d4b000594 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:39.926455 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc165f49-11dc-41f7-a164-dc9d4b000594 None None] PATH_INFO: `/v3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:39.930672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1/1] 10.222.0.50 () {50 vars in 791 bytes} [Fri Apr 29 18:28:39 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:28:47.895339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a6c82-212e-4681-a300-5970cfc43b10 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:47.895952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a6c82-212e-4681-a300-5970cfc43b10 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:47.895952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a6c82-212e-4681-a300-5970cfc43b10 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:47.900118 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1/2] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:28:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:28:47.906386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d13f1-4fff-4a76-9410-2b3d2ccbb855 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:47.907603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d13f1-4fff-4a76-9410-2b3d2ccbb855 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:47.908252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d13f1-4fff-4a76-9410-2b3d2ccbb855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:47.941224 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG oslo_db.sqlalchemy.engines [None req-fc0d13f1-4fff-4a76-9410-2b3d2ccbb855 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80132) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Apr 29 18:28:48.123906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG passlib.handlers.bcrypt [None req-fc0d13f1-4fff-4a76-9410-2b3d2ccbb855 None None] detected 'bcrypt' backend, version '3.2.0' {{(pid=80132) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Apr 29 18:28:48.124324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG passlib.handlers.bcrypt [None req-fc0d13f1-4fff-4a76-9410-2b3d2ccbb855 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80132) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Apr 29 18:28:48.179869 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-fc0d13f1-4fff-4a76-9410-2b3d2ccbb855 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:28:48.181420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fc0d13f1-4fff-4a76-9410-2b3d2ccbb855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:28:48.312193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 2/3] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:28:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Apr 29 18:28:48.445938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a6c82-212e-4681-a300-5970cfc43b10 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:48.481421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG oslo_db.sqlalchemy.engines [None req-9b5a6c82-212e-4681-a300-5970cfc43b10 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80131) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Apr 29 18:28:48.606690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a6c82-212e-4681-a300-5970cfc43b10 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:48.611437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297e1f8-f965-4a13-a282-3da260b885b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:48.612106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297e1f8-f965-4a13-a282-3da260b885b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:48.612391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297e1f8-f965-4a13-a282-3da260b885b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:48.612682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297e1f8-f965-4a13-a282-3da260b885b8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:48.622188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297e1f8-f965-4a13-a282-3da260b885b8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:48.674871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d297e1f8-f965-4a13-a282-3da260b885b8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Apr 29 18:28:48.675185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297e1f8-f965-4a13-a282-3da260b885b8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:48.686323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 2/4] 10.222.0.50 () {58 vars in 1324 bytes} [Fri Apr 29 18:28:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:28:48.692534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d13f1-4fff-4a76-9410-2b3d2ccbb855 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:48.707729 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d13f1-4fff-4a76-9410-2b3d2ccbb855 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:48.712710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd36db5-e27b-457a-a763-49c425207607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:48.714125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd36db5-e27b-457a-a763-49c425207607 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:48.714512 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd36db5-e27b-457a-a763-49c425207607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:48.714795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd36db5-e27b-457a-a763-49c425207607 admin admin] PATH_INFO: `/v3/projects/28300f353c4c4c27a229238b2a238c8e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:48.716508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd36db5-e27b-457a-a763-49c425207607 admin admin] RBAC: Authorizing `identity:get_project(project_id=28300f353c4c4c27a229238b2a238c8e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:48.767888 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6cd36db5-e27b-457a-a763-49c425207607 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Apr 29 18:28:48.768205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd36db5-e27b-457a-a763-49c425207607 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:48.773044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 3/5] 10.222.0.50 () {56 vars in 1207 bytes} [Fri Apr 29 18:28:48 2022] GET /identity/v3/projects/28300f353c4c4c27a229238b2a238c8e => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:28:51.055267 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ce46c-4cef-4d77-bd80-aa06e1ad70e3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:51.055867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ce46c-4cef-4d77-bd80-aa06e1ad70e3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:51.055867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ce46c-4cef-4d77-bd80-aa06e1ad70e3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:51.060062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 3/6] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:28:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:28:51.065787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934f184-b96f-49d2-8ca8-e492044cd654 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:51.066210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934f184-b96f-49d2-8ca8-e492044cd654 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:51.066306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934f184-b96f-49d2-8ca8-e492044cd654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:51.087123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3934f184-b96f-49d2-8ca8-e492044cd654 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:28:51.088856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3934f184-b96f-49d2-8ca8-e492044cd654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:28:51.097969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 4/7] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:28:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Apr 29 18:28:51.121330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ce46c-4cef-4d77-bd80-aa06e1ad70e3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:51.135397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ce46c-4cef-4d77-bd80-aa06e1ad70e3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:51.140469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148068ca-6206-4c5e-9eba-3ba8f85af94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:51.141141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148068ca-6206-4c5e-9eba-3ba8f85af94d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:51.141421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148068ca-6206-4c5e-9eba-3ba8f85af94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:51.141693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148068ca-6206-4c5e-9eba-3ba8f85af94d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:51.149569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148068ca-6206-4c5e-9eba-3ba8f85af94d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:51.151421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148068ca-6206-4c5e-9eba-3ba8f85af94d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:51.160134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 4/8] 10.222.0.50 () {58 vars in 1324 bytes} [Fri Apr 29 18:28:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:28:51.165785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934f184-b96f-49d2-8ca8-e492044cd654 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:51.175388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934f184-b96f-49d2-8ca8-e492044cd654 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:51.181113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807f70f-4815-43c5-b2c5-4e21f9d01f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:51.182090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807f70f-4815-43c5-b2c5-4e21f9d01f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:51.182373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807f70f-4815-43c5-b2c5-4e21f9d01f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:51.182647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807f70f-4815-43c5-b2c5-4e21f9d01f2b admin admin] PATH_INFO: `/v3/users/a74b9eac8d87492aad683f6da38fa8b9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:51.184197 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807f70f-4815-43c5-b2c5-4e21f9d01f2b admin admin] RBAC: Authorizing `identity:get_user(user_id=a74b9eac8d87492aad683f6da38fa8b9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:51.186085 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807f70f-4815-43c5-b2c5-4e21f9d01f2b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:51.187683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 5/9] 10.222.0.50 () {56 vars in 1198 bytes} [Fri Apr 29 18:28:51 2022] GET /identity/v3/users/a74b9eac8d87492aad683f6da38fa8b9 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:28:53.605226 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371f622-041d-49c6-8554-61fec930c08d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:53.605770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371f622-041d-49c6-8554-61fec930c08d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:53.605770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371f622-041d-49c6-8554-61fec930c08d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:53.607137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 5/10] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:28:53.612521 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8f4c1-ee6e-4621-b1d7-7e8f7c2e559e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:53.612965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8f4c1-ee6e-4621-b1d7-7e8f7c2e559e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:53.613057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8f4c1-ee6e-4621-b1d7-7e8f7c2e559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:53.615863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00c191-8c92-4e1d-ab52-cc10b215bede None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:53.616154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00c191-8c92-4e1d-ab52-cc10b215bede None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:53.616464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00c191-8c92-4e1d-ab52-cc10b215bede None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:53.620322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 6/11] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:28:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:28:53.622974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48d1f9-61df-4c39-9ffb-6b26e7437e1d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:53.623265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48d1f9-61df-4c39-9ffb-6b26e7437e1d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:53.623540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48d1f9-61df-4c39-9ffb-6b26e7437e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:53.664923 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG passlib.handlers.bcrypt [None req-5b48d1f9-61df-4c39-9ffb-6b26e7437e1d None None] detected 'bcrypt' backend, version '3.2.0' {{(pid=80131) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Apr 29 18:28:53.665325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG passlib.handlers.bcrypt [None req-5b48d1f9-61df-4c39-9ffb-6b26e7437e1d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80131) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Apr 29 18:28:53.712264 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5b48d1f9-61df-4c39-9ffb-6b26e7437e1d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:28:53.714049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5b48d1f9-61df-4c39-9ffb-6b26e7437e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:28:53.743726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-99e8f4c1-ee6e-4621-b1d7-7e8f7c2e559e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:28:53.745014 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-99e8f4c1-ee6e-4621-b1d7-7e8f7c2e559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:28:53.824922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 6/12] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:28:53.828342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 7/13] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:28:53.848473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8f4c1-ee6e-4621-b1d7-7e8f7c2e559e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:53.849856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48d1f9-61df-4c39-9ffb-6b26e7437e1d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:53.949860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48d1f9-61df-4c39-9ffb-6b26e7437e1d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:53.955131 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8f4c1-ee6e-4621-b1d7-7e8f7c2e559e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:53.984028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5308ad8-1b57-4fa6-b0c9-413e2c8604f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:53.984974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308ad8-1b57-4fa6-b0c9-413e2c8604f1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:53.985259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308ad8-1b57-4fa6-b0c9-413e2c8604f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:53.985534 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308ad8-1b57-4fa6-b0c9-413e2c8604f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:53.998443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229bcba-3ed3-4dcc-b24d-f2c8fc1b01c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:53.999221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229bcba-3ed3-4dcc-b24d-f2c8fc1b01c0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:53.999529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229bcba-3ed3-4dcc-b24d-f2c8fc1b01c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:53.999798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229bcba-3ed3-4dcc-b24d-f2c8fc1b01c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:54.010578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229bcba-3ed3-4dcc-b24d-f2c8fc1b01c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d229bcba-3ed3-4dcc-b24d-f2c8fc1b01c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:28:54.021492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:28:54.022605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:28:54.023626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 7/14] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:28:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:28:54.028895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229bcba-3ed3-4dcc-b24d-f2c8fc1b01c0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:54.054516 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5308ad8-1b57-4fa6-b0c9-413e2c8604f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:54.056041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5308ad8-1b57-4fa6-b0c9-413e2c8604f1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:54.124078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229bcba-3ed3-4dcc-b24d-f2c8fc1b01c0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:54.153437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 8/15] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:28:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:28:54.157170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d77f7-1c0e-418f-b739-09594ba58672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:54.157845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d77f7-1c0e-418f-b739-09594ba58672 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:54.158193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d77f7-1c0e-418f-b739-09594ba58672 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:54.158479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d77f7-1c0e-418f-b739-09594ba58672 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:54.159221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d77f7-1c0e-418f-b739-09594ba58672 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:28:54.159307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5308ad8-1b57-4fa6-b0c9-413e2c8604f1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:54.159518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d77f7-1c0e-418f-b739-09594ba58672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:54.160948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d77f7-1c0e-418f-b739-09594ba58672 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:54.173810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 8/16] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:28:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:28:54.179894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d77f7-1c0e-418f-b739-09594ba58672 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:54.292364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d77f7-1c0e-418f-b739-09594ba58672 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:54.327603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d3f0e-547a-4290-b810-ac559c09b95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:54.328465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d3f0e-547a-4290-b810-ac559c09b95a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:54.328782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d3f0e-547a-4290-b810-ac559c09b95a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:54.329055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d3f0e-547a-4290-b810-ac559c09b95a None admin] PATH_INFO: `/v3/users/a74b9eac8d87492aad683f6da38fa8b9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:54.330264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d3f0e-547a-4290-b810-ac559c09b95a None admin] RBAC: Authorizing `identity:get_user(user_id=a74b9eac8d87492aad683f6da38fa8b9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:54.331734 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d3f0e-547a-4290-b810-ac559c09b95a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:54.333372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 9/17] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:28:54 2022] GET /identity/v3/users/a74b9eac8d87492aad683f6da38fa8b9 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:28:54.339421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d3f0e-547a-4290-b810-ac559c09b95a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:54.382184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5308ad8-1b57-4fa6-b0c9-413e2c8604f1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:54.413711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687821d-23d5-4cc0-8810-4be260595449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:54.414500 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687821d-23d5-4cc0-8810-4be260595449 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:54.414781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687821d-23d5-4cc0-8810-4be260595449 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:54.415043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687821d-23d5-4cc0-8810-4be260595449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:54.434546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687821d-23d5-4cc0-8810-4be260595449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:54.436036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687821d-23d5-4cc0-8810-4be260595449 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:54.439352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d3f0e-547a-4290-b810-ac559c09b95a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4687821d-23d5-4cc0-8810-4be260595449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:28:54.456791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:28:54.457639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:28:54.458441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:28:54.458441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:28:54.458441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:28:54.458441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:28:54.460681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 9/18] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:28:54.463757 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687821d-23d5-4cc0-8810-4be260595449 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:54.472702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df513cf-ad33-4fd4-ab6e-183dddb4ac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:54.473517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df513cf-ad33-4fd4-ab6e-183dddb4ac0f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:54.473796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df513cf-ad33-4fd4-ab6e-183dddb4ac0f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:54.474068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df513cf-ad33-4fd4-ab6e-183dddb4ac0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:54.475360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df513cf-ad33-4fd4-ab6e-183dddb4ac0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:54.476893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df513cf-ad33-4fd4-ab6e-183dddb4ac0f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:54.478488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 10/19] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:28:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:28:54.484295 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df513cf-ad33-4fd4-ab6e-183dddb4ac0f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:54.562413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687821d-23d5-4cc0-8810-4be260595449 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:54.581739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df513cf-ad33-4fd4-ab6e-183dddb4ac0f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:54.600206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60967da0-64df-4de2-a647-41d4a993ef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:54.600900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60967da0-64df-4de2-a647-41d4a993ef47 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:54.601151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60967da0-64df-4de2-a647-41d4a993ef47 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:54.601446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60967da0-64df-4de2-a647-41d4a993ef47 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:54.602161 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60967da0-64df-4de2-a647-41d4a993ef47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:28:54.602464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60967da0-64df-4de2-a647-41d4a993ef47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:54.604234 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60967da0-64df-4de2-a647-41d4a993ef47 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:54.614973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ab60f-ba4a-4dff-9ab9-b20377ffc5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:54.615661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab60f-ba4a-4dff-9ab9-b20377ffc5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:54.615945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab60f-ba4a-4dff-9ab9-b20377ffc5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:54.616215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab60f-ba4a-4dff-9ab9-b20377ffc5d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:54.617081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ab60f-ba4a-4dff-9ab9-b20377ffc5d3 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.domain.id=default, user.id=a74b9eac8d87492aad683f6da38fa8b9) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:28:54.617388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ab60f-ba4a-4dff-9ab9-b20377ffc5d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:54.618874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ab60f-ba4a-4dff-9ab9-b20377ffc5d3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:54.622285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 10/20] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:28:54.626124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 11/21] 10.222.0.50 () {56 vars in 1322 bytes} [Fri Apr 29 18:28:54 2022] GET /identity/v3/role_assignments?user.id=a74b9eac8d87492aad683f6da38fa8b9&scope.domain.id=default&role.id=e55ef11844a742f5889243f189bd59bc => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:28:57.256889 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a08f28-ff59-4a6b-9f29-c9cdcaf64bbd None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:57.257928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a08f28-ff59-4a6b-9f29-c9cdcaf64bbd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:57.258440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a08f28-ff59-4a6b-9f29-c9cdcaf64bbd None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:57.261274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 11/22] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:28:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:28:57.270847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f500b9f-e5a7-4537-b958-51f9e8a39465 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:57.271781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f500b9f-e5a7-4537-b958-51f9e8a39465 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:57.272247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f500b9f-e5a7-4537-b958-51f9e8a39465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:57.294849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0f500b9f-e5a7-4537-b958-51f9e8a39465 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:28:57.296388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0f500b9f-e5a7-4537-b958-51f9e8a39465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:28:57.373836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 12/23] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:28:57.402820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a08f28-ff59-4a6b-9f29-c9cdcaf64bbd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:57.456121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64d424-bb62-477d-80bb-4e439b216f28 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:57.456778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64d424-bb62-477d-80bb-4e439b216f28 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:57.457250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64d424-bb62-477d-80bb-4e439b216f28 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:57.459101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 13/24] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:28:57.465292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad1932-4699-47f8-9954-9b4cbb435eda None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:57.465837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad1932-4699-47f8-9954-9b4cbb435eda None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:57.466306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad1932-4699-47f8-9954-9b4cbb435eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:57.497573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5cad1932-4699-47f8-9954-9b4cbb435eda None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:28:57.498766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5cad1932-4699-47f8-9954-9b4cbb435eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:28:57.554274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a08f28-ff59-4a6b-9f29-c9cdcaf64bbd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:57.611049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd8c95-3878-4143-b186-702412d4ebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:57.611873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd8c95-3878-4143-b186-702412d4ebe1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:57.612152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd8c95-3878-4143-b186-702412d4ebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:57.612421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd8c95-3878-4143-b186-702412d4ebe1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:57.649443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd8c95-3878-4143-b186-702412d4ebe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8dfd8c95-3878-4143-b186-702412d4ebe1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:28:57.658332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:28:57.659191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:28:57.659943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:28:57.664988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 12/25] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:28:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:28:57.666520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 14/26] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:28:57.671074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd8c95-3878-4143-b186-702412d4ebe1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:57.714074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad1932-4699-47f8-9954-9b4cbb435eda None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:57.878950 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd8c95-3878-4143-b186-702412d4ebe1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:57.919526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad1932-4699-47f8-9954-9b4cbb435eda None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:57.942194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e138-0deb-4fcf-a87a-40519e34f370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:57.942896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e138-0deb-4fcf-a87a-40519e34f370 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:57.943174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e138-0deb-4fcf-a87a-40519e34f370 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:57.943451 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e138-0deb-4fcf-a87a-40519e34f370 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:57.944211 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502e138-0deb-4fcf-a87a-40519e34f370 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:28:57.944540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502e138-0deb-4fcf-a87a-40519e34f370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:57.945958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502e138-0deb-4fcf-a87a-40519e34f370 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:57.971056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 13/27] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:28:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:28:57.973254 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe7884-0240-476d-9ab6-dc3e8479417a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:57.973846 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe7884-0240-476d-9ab6-dc3e8479417a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:57.974735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe7884-0240-476d-9ab6-dc3e8479417a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:57.976540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 14/28] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:28:57.979469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe7884-0240-476d-9ab6-dc3e8479417a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:57.981788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a64a64-b819-4dfb-bb72-a4afca61d977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:57.982774 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a64a64-b819-4dfb-bb72-a4afca61d977 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:57.983192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a64a64-b819-4dfb-bb72-a4afca61d977 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:57.983596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a64a64-b819-4dfb-bb72-a4afca61d977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.005496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a64a64-b819-4dfb-bb72-a4afca61d977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:58.006979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a64a64-b819-4dfb-bb72-a4afca61d977 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-25a64a64-b819-4dfb-bb72-a4afca61d977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:28:58.036318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:28:58.037229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:28:58.038097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:28:58.038097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:28:58.038097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:28:58.038097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:28:58.043442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 15/29] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:28:58.052624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92446a-48d0-4cac-bcb6-316c1f701723 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.052909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92446a-48d0-4cac-bcb6-316c1f701723 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.053179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92446a-48d0-4cac-bcb6-316c1f701723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.072849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-cd92446a-48d0-4cac-bcb6-316c1f701723 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:28:58.074229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cd92446a-48d0-4cac-bcb6-316c1f701723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:28:58.126946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe7884-0240-476d-9ab6-dc3e8479417a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:58.157901 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 16/30] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:28:58.158752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92446a-48d0-4cac-bcb6-316c1f701723 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:58.160878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629eb80-69c3-4c0c-bf12-ab8e8175cfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:58.161832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629eb80-69c3-4c0c-bf12-ab8e8175cfbb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.162181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629eb80-69c3-4c0c-bf12-ab8e8175cfbb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.162552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629eb80-69c3-4c0c-bf12-ab8e8175cfbb None admin] PATH_INFO: `/v3/users/a74b9eac8d87492aad683f6da38fa8b9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.164203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629eb80-69c3-4c0c-bf12-ab8e8175cfbb None admin] RBAC: Authorizing `identity:get_user(user_id=a74b9eac8d87492aad683f6da38fa8b9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:58.165882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629eb80-69c3-4c0c-bf12-ab8e8175cfbb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:58.168320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 15/31] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:28:57 2022] GET /identity/v3/users/a74b9eac8d87492aad683f6da38fa8b9 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:28:58.169830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad5281-6397-4ab6-81d4-99b9f1be400a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.170220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad5281-6397-4ab6-81d4-99b9f1be400a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.170593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad5281-6397-4ab6-81d4-99b9f1be400a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.172284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 16/32] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:28:58.177150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad5281-6397-4ab6-81d4-99b9f1be400a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:58.274736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92446a-48d0-4cac-bcb6-316c1f701723 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:58.292534 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad5281-6397-4ab6-81d4-99b9f1be400a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:58.309847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51b5a7-3530-4bf6-a8a1-32306431ec7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:58.310702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51b5a7-3530-4bf6-a8a1-32306431ec7d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.311119 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51b5a7-3530-4bf6-a8a1-32306431ec7d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.311534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51b5a7-3530-4bf6-a8a1-32306431ec7d None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.312367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51b5a7-3530-4bf6-a8a1-32306431ec7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:28:58.312817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51b5a7-3530-4bf6-a8a1-32306431ec7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:58.314378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51b5a7-3530-4bf6-a8a1-32306431ec7d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:58.327973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb738c9d-c846-40db-8e6e-4ef244545be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:58.329090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb738c9d-c846-40db-8e6e-4ef244545be2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.329503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb738c9d-c846-40db-8e6e-4ef244545be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.329920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb738c9d-c846-40db-8e6e-4ef244545be2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.331240 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb738c9d-c846-40db-8e6e-4ef244545be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:58.332867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb738c9d-c846-40db-8e6e-4ef244545be2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:58.338978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 17/33] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:28:58.338978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267136b-e6fa-43d8-a23b-d4e2878e8857 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.338978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267136b-e6fa-43d8-a23b-d4e2878e8857 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.338978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267136b-e6fa-43d8-a23b-d4e2878e8857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.339484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 17/34] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:28:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:28:58.340551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb738c9d-c846-40db-8e6e-4ef244545be2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:58.354629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5267136b-e6fa-43d8-a23b-d4e2878e8857 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:28:58.355953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5267136b-e6fa-43d8-a23b-d4e2878e8857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:28:58.426624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 18/35] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:28:58.426624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267136b-e6fa-43d8-a23b-d4e2878e8857 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:58.446197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb738c9d-c846-40db-8e6e-4ef244545be2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:58.481010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82db580-1e07-41e5-a696-125dce47fc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:58.481712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82db580-1e07-41e5-a696-125dce47fc0a None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.481973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82db580-1e07-41e5-a696-125dce47fc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.482531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82db580-1e07-41e5-a696-125dce47fc0a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.483283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82db580-1e07-41e5-a696-125dce47fc0a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:58.484695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82db580-1e07-41e5-a696-125dce47fc0a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:58.523398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267136b-e6fa-43d8-a23b-d4e2878e8857 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:58.558446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27698497-f82f-49e2-a4b3-c7656f4ea1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:58.559333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27698497-f82f-49e2-a4b3-c7656f4ea1c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.559623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27698497-f82f-49e2-a4b3-c7656f4ea1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.559904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27698497-f82f-49e2-a4b3-c7656f4ea1c5 None admin] PATH_INFO: `/v3/domains/default/users/a74b9eac8d87492aad683f6da38fa8b9/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.561924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27698497-f82f-49e2-a4b3-c7656f4ea1c5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a74b9eac8d87492aad683f6da38fa8b9, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:58.563381 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27698497-f82f-49e2-a4b3-c7656f4ea1c5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:58.607187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 18/36] 10.222.0.50 () {60 vars in 1131 bytes} [Fri Apr 29 18:28:58 2022] POST /identity/v3/roles => generated 219 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:28:58.608161 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82db580-1e07-41e5-a696-125dce47fc0a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:58.712662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 19/37] 10.222.0.50 () {58 vars in 1361 bytes} [Fri Apr 29 18:28:58 2022] PUT /identity/v3/domains/default/users/a74b9eac8d87492aad683f6da38fa8b9/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:28:58.715839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5b946-67ac-4348-9197-dd3dc5eba47a None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.716219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5b946-67ac-4348-9197-dd3dc5eba47a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.716590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5b946-67ac-4348-9197-dd3dc5eba47a None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.721102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82db580-1e07-41e5-a696-125dce47fc0a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:58.722909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 20/38] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:28:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:28:58.724776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5b946-67ac-4348-9197-dd3dc5eba47a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:58.767112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b2ded-36ae-4158-a78b-b1bb18a87869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:58.767815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b2ded-36ae-4158-a78b-b1bb18a87869 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.768191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b2ded-36ae-4158-a78b-b1bb18a87869 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.768471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b2ded-36ae-4158-a78b-b1bb18a87869 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.769261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b2ded-36ae-4158-a78b-b1bb18a87869 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:58.772775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b2ded-36ae-4158-a78b-b1bb18a87869 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:58.828067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 19/39] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:28:58 2022] POST /identity/v3/projects => generated 301 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:28:58.830661 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02f396-0eff-4c2c-9fbf-fd1154bdd425 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.830950 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02f396-0eff-4c2c-9fbf-fd1154bdd425 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.831228 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02f396-0eff-4c2c-9fbf-fd1154bdd425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.867779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7b02f396-0eff-4c2c-9fbf-fd1154bdd425 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:28:58.875142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7b02f396-0eff-4c2c-9fbf-fd1154bdd425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:28:58.931648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5b946-67ac-4348-9197-dd3dc5eba47a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:58.983496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9f768-179c-4204-a6cb-76dd740d0c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:58.984112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9f768-179c-4204-a6cb-76dd740d0c5a None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:58.984397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9f768-179c-4204-a6cb-76dd740d0c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:58.984733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9f768-179c-4204-a6cb-76dd740d0c5a None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:58.985505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9f768-179c-4204-a6cb-76dd740d0c5a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:58.987066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9f768-179c-4204-a6cb-76dd740d0c5a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:58.999816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 20/40] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:28:59.008395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 21/41] 10.222.0.50 () {60 vars in 1131 bytes} [Fri Apr 29 18:28:58 2022] POST /identity/v3/roles => generated 225 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:28:59.026709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02f396-0eff-4c2c-9fbf-fd1154bdd425 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:59.136783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02f396-0eff-4c2c-9fbf-fd1154bdd425 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:59.170889 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78bc82-1ab4-40e8-b7cb-75a6f4a981aa None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:59.171330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78bc82-1ab4-40e8-b7cb-75a6f4a981aa None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:59.171724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78bc82-1ab4-40e8-b7cb-75a6f4a981aa None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:59.173267 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 22/42] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:28:59.175529 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44b2f6-80e5-4a23-b4fc-939f9e073419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:59.176620 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44b2f6-80e5-4a23-b4fc-939f9e073419 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:59.177048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44b2f6-80e5-4a23-b4fc-939f9e073419 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:59.177455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44b2f6-80e5-4a23-b4fc-939f9e073419 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:59.178497 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44b2f6-80e5-4a23-b4fc-939f9e073419 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:59.180077 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44b2f6-80e5-4a23-b4fc-939f9e073419 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:59.180387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1aca-1310-4583-adac-9a505472bd04 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:59.180799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1aca-1310-4583-adac-9a505472bd04 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:59.181195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1aca-1310-4583-adac-9a505472bd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:59.198127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 21/43] 10.222.0.50 () {60 vars in 1131 bytes} [Fri Apr 29 18:28:59 2022] POST /identity/v3/roles => generated 223 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:28:59.205997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1d2b1aca-1310-4583-adac-9a505472bd04 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:28:59.208084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1d2b1aca-1310-4583-adac-9a505472bd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:28:59.289639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 23/44] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:28:59.312879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44b2f6-80e5-4a23-b4fc-939f9e073419 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:59.424867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44b2f6-80e5-4a23-b4fc-939f9e073419 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:59.457172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb11fd-2651-46bd-a26b-348091670922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:59.457966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb11fd-2651-46bd-a26b-348091670922 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:59.458249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb11fd-2651-46bd-a26b-348091670922 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:59.458522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb11fd-2651-46bd-a26b-348091670922 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:59.459702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb11fd-2651-46bd-a26b-348091670922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:59.461249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb11fd-2651-46bd-a26b-348091670922 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:59.462918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 22/45] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:28:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:28:59.469549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b1aca-1310-4583-adac-9a505472bd04 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:28:59.563709 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b1aca-1310-4583-adac-9a505472bd04 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:28:59.598330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70e51e-f7b3-46ae-a434-b6c261b19756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:28:59.599012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e51e-f7b3-46ae-a434-b6c261b19756 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:28:59.599271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e51e-f7b3-46ae-a434-b6c261b19756 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:28:59.599572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e51e-f7b3-46ae-a434-b6c261b19756 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:28:59.600281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70e51e-f7b3-46ae-a434-b6c261b19756 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:28:59.601785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70e51e-f7b3-46ae-a434-b6c261b19756 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:28:59.634406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 24/46] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:28:59 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:01.175065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb266f33-f5bf-4dac-b639-780ecfee0986 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:01.175587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb266f33-f5bf-4dac-b639-780ecfee0986 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:01.175735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb266f33-f5bf-4dac-b639-780ecfee0986 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:01.177181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 23/47] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:01.182610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebfb3e-0b77-42b7-9841-ff5ef02ca8e1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:01.183035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebfb3e-0b77-42b7-9841-ff5ef02ca8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:01.183147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebfb3e-0b77-42b7-9841-ff5ef02ca8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:01.201462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0eebfb3e-0b77-42b7-9841-ff5ef02ca8e1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:01.202668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0eebfb3e-0b77-42b7-9841-ff5ef02ca8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:01.271358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 25/48] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:01.295031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb266f33-f5bf-4dac-b639-780ecfee0986 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:01.400875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb266f33-f5bf-4dac-b639-780ecfee0986 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:01.434631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d261fa8-323b-43bb-97ad-1981128a98d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:01.435363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d261fa8-323b-43bb-97ad-1981128a98d3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:01.435653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d261fa8-323b-43bb-97ad-1981128a98d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:01.435927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d261fa8-323b-43bb-97ad-1981128a98d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:01.446142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d261fa8-323b-43bb-97ad-1981128a98d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3d261fa8-323b-43bb-97ad-1981128a98d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:01.453929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:01.454952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:01.455860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 24/49] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:29:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:01.462706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebfb3e-0b77-42b7-9841-ff5ef02ca8e1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:01.561285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebfb3e-0b77-42b7-9841-ff5ef02ca8e1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:01.595661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5ede5-72c3-4e89-9055-a024a16423c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:01.596331 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5ede5-72c3-4e89-9055-a024a16423c9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:01.596662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5ede5-72c3-4e89-9055-a024a16423c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:01.596913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5ede5-72c3-4e89-9055-a024a16423c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:01.597689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5ede5-72c3-4e89-9055-a024a16423c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:01.597998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5ede5-72c3-4e89-9055-a024a16423c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:01.599410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5ede5-72c3-4e89-9055-a024a16423c9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:01.607913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 26/50] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:01.613850 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d261fa8-323b-43bb-97ad-1981128a98d3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:01.715170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d261fa8-323b-43bb-97ad-1981128a98d3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:01.763685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac64a-6936-4abf-a50b-4110594e2c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:01.764622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac64a-6936-4abf-a50b-4110594e2c1a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:01.764908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac64a-6936-4abf-a50b-4110594e2c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:01.765184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac64a-6936-4abf-a50b-4110594e2c1a None admin] PATH_INFO: `/v3/users/a74b9eac8d87492aad683f6da38fa8b9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:01.766467 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ac64a-6936-4abf-a50b-4110594e2c1a None admin] RBAC: Authorizing `identity:get_user(user_id=a74b9eac8d87492aad683f6da38fa8b9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:01.767959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ac64a-6936-4abf-a50b-4110594e2c1a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:01.769756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 25/51] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:01 2022] GET /identity/v3/users/a74b9eac8d87492aad683f6da38fa8b9 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:01.775446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5ede5-72c3-4e89-9055-a024a16423c9 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:01.875213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5ede5-72c3-4e89-9055-a024a16423c9 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:01.910652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049172a-5dff-424d-8487-671f869ed255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:01.911462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049172a-5dff-424d-8487-671f869ed255 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:01.911745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049172a-5dff-424d-8487-671f869ed255 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:01.912019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049172a-5dff-424d-8487-671f869ed255 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:01.913265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049172a-5dff-424d-8487-671f869ed255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:01.914733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049172a-5dff-424d-8487-671f869ed255 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:01.916481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 27/52] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:01.922451 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac64a-6936-4abf-a50b-4110594e2c1a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:01.923941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe7616-19dd-47d8-b853-b248be0f5350 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:01.924232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe7616-19dd-47d8-b853-b248be0f5350 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:01.924537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe7616-19dd-47d8-b853-b248be0f5350 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:01.925810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 28/53] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:29:01.930694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e07ef-4f54-4c1a-bc55-ea49c629009f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:01.930980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e07ef-4f54-4c1a-bc55-ea49c629009f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:01.931262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e07ef-4f54-4c1a-bc55-ea49c629009f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:01.948217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ea2e07ef-4f54-4c1a-bc55-ea49c629009f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:01.949433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ea2e07ef-4f54-4c1a-bc55-ea49c629009f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:02.016454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 29/54] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:02.023066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac64a-6936-4abf-a50b-4110594e2c1a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:02.040235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e07ef-4f54-4c1a-bc55-ea49c629009f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:02.071163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90b80a-8e22-4c95-b704-8090dd7d0aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:02.073334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90b80a-8e22-4c95-b704-8090dd7d0aea None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:02.073334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90b80a-8e22-4c95-b704-8090dd7d0aea None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:02.073334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90b80a-8e22-4c95-b704-8090dd7d0aea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:02.074825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90b80a-8e22-4c95-b704-8090dd7d0aea None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.domain.id=default, user.id=a74b9eac8d87492aad683f6da38fa8b9) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:02.076009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90b80a-8e22-4c95-b704-8090dd7d0aea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:02.077181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90b80a-8e22-4c95-b704-8090dd7d0aea None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:02.088169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 26/55] 10.222.0.50 () {56 vars in 1322 bytes} [Fri Apr 29 18:29:01 2022] GET /identity/v3/role_assignments?user.id=a74b9eac8d87492aad683f6da38fa8b9&scope.domain.id=default&role.id=e55ef11844a742f5889243f189bd59bc => generated 530 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:02.153421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e07ef-4f54-4c1a-bc55-ea49c629009f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:02.191267 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eb8c4-37a5-4569-8601-eb5277c8764e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:02.192101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eb8c4-37a5-4569-8601-eb5277c8764e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:02.192389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eb8c4-37a5-4569-8601-eb5277c8764e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:02.192702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eb8c4-37a5-4569-8601-eb5277c8764e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:02.194066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eb8c4-37a5-4569-8601-eb5277c8764e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:02.195563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eb8c4-37a5-4569-8601-eb5277c8764e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:02.197352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 30/56] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:02.203727 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90b80a-8e22-4c95-b704-8090dd7d0aea None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:02.307832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90b80a-8e22-4c95-b704-8090dd7d0aea None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:02.340155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c15a2-7f40-4d3d-86e1-0e0294aabe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:02.343692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c15a2-7f40-4d3d-86e1-0e0294aabe23 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:02.343692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c15a2-7f40-4d3d-86e1-0e0294aabe23 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:02.343692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c15a2-7f40-4d3d-86e1-0e0294aabe23 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:02.343692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c15a2-7f40-4d3d-86e1-0e0294aabe23 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:02.343692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c15a2-7f40-4d3d-86e1-0e0294aabe23 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:02.378141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 27/57] 10.222.0.50 () {60 vars in 1140 bytes} [Fri Apr 29 18:29:02 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:04.612374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5970600-39c5-4458-a8a6-d0c357182ba1 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:04.612959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5970600-39c5-4458-a8a6-d0c357182ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:04.612959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5970600-39c5-4458-a8a6-d0c357182ba1 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:04.614306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 31/58] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:04.620196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c0b9-a878-4026-951c-7223b1554382 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:04.620643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c0b9-a878-4026-951c-7223b1554382 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:04.620759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7c0b9-a878-4026-951c-7223b1554382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:04.638326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0ed7c0b9-a878-4026-951c-7223b1554382 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:04.639533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0ed7c0b9-a878-4026-951c-7223b1554382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:04.720174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 28/59] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:04.744433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5970600-39c5-4458-a8a6-d0c357182ba1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:04.845395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5970600-39c5-4458-a8a6-d0c357182ba1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:04.880726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b57b60-4ce5-43ae-97df-5e1d18b70be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:04.881443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b57b60-4ce5-43ae-97df-5e1d18b70be8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:04.881722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b57b60-4ce5-43ae-97df-5e1d18b70be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:04.881986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b57b60-4ce5-43ae-97df-5e1d18b70be8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:04.883268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b57b60-4ce5-43ae-97df-5e1d18b70be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:04.884823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b57b60-4ce5-43ae-97df-5e1d18b70be8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:04.886401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 32/60] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:04.892262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7c0b9-a878-4026-951c-7223b1554382 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:04.997739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7c0b9-a878-4026-951c-7223b1554382 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:05.031078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d7265-0ac5-4fa7-9b0e-c7434637bbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:05.031754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d7265-0ac5-4fa7-9b0e-c7434637bbe2 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:05.032030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d7265-0ac5-4fa7-9b0e-c7434637bbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:05.032301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d7265-0ac5-4fa7-9b0e-c7434637bbe2 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:05.033040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d7265-0ac5-4fa7-9b0e-c7434637bbe2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:05.034481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d7265-0ac5-4fa7-9b0e-c7434637bbe2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:05.068824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 29/61] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:29:04 2022] POST /identity/v3/users => generated 275 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:08.436650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca16905-2130-4e69-8e2c-9f1dd611cdc8 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:08.437743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca16905-2130-4e69-8e2c-9f1dd611cdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:08.438233 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca16905-2130-4e69-8e2c-9f1dd611cdc8 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:08.439792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 33/62] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:08.445103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24912d-66e8-47a9-892a-27ba3a9cd7d6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:08.445913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24912d-66e8-47a9-892a-27ba3a9cd7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:08.446403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24912d-66e8-47a9-892a-27ba3a9cd7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:08.469930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8c24912d-66e8-47a9-892a-27ba3a9cd7d6 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:08.471469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8c24912d-66e8-47a9-892a-27ba3a9cd7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:08.554785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 30/63] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:08.585866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca16905-2130-4e69-8e2c-9f1dd611cdc8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:08.626779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987d882-3bbf-4c44-973e-6cfe4c35563f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:08.627355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987d882-3bbf-4c44-973e-6cfe4c35563f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:08.627826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987d882-3bbf-4c44-973e-6cfe4c35563f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:08.629684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 31/64] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:08.635276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26905bbe-4782-44d1-ba3b-c8627de0e436 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:08.635943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26905bbe-4782-44d1-ba3b-c8627de0e436 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:08.636455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26905bbe-4782-44d1-ba3b-c8627de0e436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:08.674796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-26905bbe-4782-44d1-ba3b-c8627de0e436 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:08.676354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26905bbe-4782-44d1-ba3b-c8627de0e436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:08.709998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca16905-2130-4e69-8e2c-9f1dd611cdc8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:08.770188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 32/65] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:08.771106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af7d29-0694-4963-b5bf-35c2f40accde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:08.772221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af7d29-0694-4963-b5bf-35c2f40accde None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:08.772773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af7d29-0694-4963-b5bf-35c2f40accde None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:08.773187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af7d29-0694-4963-b5bf-35c2f40accde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:08.784193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af7d29-0694-4963-b5bf-35c2f40accde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-b9af7d29-0694-4963-b5bf-35c2f40accde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:08.792263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:08.793180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:08.794022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:08.796464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 34/66] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:08.802385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26905bbe-4782-44d1-ba3b-c8627de0e436 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:08.815155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af7d29-0694-4963-b5bf-35c2f40accde None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:08.905966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26905bbe-4782-44d1-ba3b-c8627de0e436 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:08.929292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af7d29-0694-4963-b5bf-35c2f40accde None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:08.939598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed827e24-b98a-4aac-8bf4-42a76115ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:08.940563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed827e24-b98a-4aac-8bf4-42a76115ce5d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:08.940984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed827e24-b98a-4aac-8bf4-42a76115ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:08.941395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed827e24-b98a-4aac-8bf4-42a76115ce5d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:08.942319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed827e24-b98a-4aac-8bf4-42a76115ce5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:08.942814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed827e24-b98a-4aac-8bf4-42a76115ce5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:08.944582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed827e24-b98a-4aac-8bf4-42a76115ce5d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:08.954268 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 33/67] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:08.955849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01914e5a-bbaf-452f-9906-872fedcfc965 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:08.956283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01914e5a-bbaf-452f-9906-872fedcfc965 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:08.956769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01914e5a-bbaf-452f-9906-872fedcfc965 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:08.958407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 34/68] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:08.959528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01914e5a-bbaf-452f-9906-872fedcfc965 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:08.970677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393068ae-496a-49fc-a39b-82859e74f1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:08.971420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393068ae-496a-49fc-a39b-82859e74f1ba None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:08.971699 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393068ae-496a-49fc-a39b-82859e74f1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:08.971972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393068ae-496a-49fc-a39b-82859e74f1ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:08.983803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393068ae-496a-49fc-a39b-82859e74f1ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-393068ae-496a-49fc-a39b-82859e74f1ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:08.991831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:08.992904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:08.993757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 35/69] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:08.999166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbcc90-bfa3-4831-afc7-ac72f7e4549f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:08.999166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbcc90-bfa3-4831-afc7-ac72f7e4549f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:08.999166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbcc90-bfa3-4831-afc7-ac72f7e4549f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.012401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5bdbcc90-bfa3-4831-afc7-ac72f7e4549f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:09.013861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5bdbcc90-bfa3-4831-afc7-ac72f7e4549f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:09.081863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01914e5a-bbaf-452f-9906-872fedcfc965 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:09.088966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 36/70] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:09.090057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbcc90-bfa3-4831-afc7-ac72f7e4549f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:09.115229 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef7ddf-2034-41f4-92a6-c2a4f19bfd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:09.116397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef7ddf-2034-41f4-92a6-c2a4f19bfd17 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.116837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef7ddf-2034-41f4-92a6-c2a4f19bfd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.117241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef7ddf-2034-41f4-92a6-c2a4f19bfd17 None admin] PATH_INFO: `/v3/users/a74b9eac8d87492aad683f6da38fa8b9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.118598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef7ddf-2034-41f4-92a6-c2a4f19bfd17 None admin] RBAC: Authorizing `identity:get_user(user_id=a74b9eac8d87492aad683f6da38fa8b9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:09.120335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef7ddf-2034-41f4-92a6-c2a4f19bfd17 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:09.122192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 35/71] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:08 2022] GET /identity/v3/users/a74b9eac8d87492aad683f6da38fa8b9 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:09.127586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef7ddf-2034-41f4-92a6-c2a4f19bfd17 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:09.209312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbcc90-bfa3-4831-afc7-ac72f7e4549f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:09.229879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef7ddf-2034-41f4-92a6-c2a4f19bfd17 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:09.242425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98f49-68bb-4bec-83b8-400e470a87eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:09.243106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98f49-68bb-4bec-83b8-400e470a87eb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.243389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98f49-68bb-4bec-83b8-400e470a87eb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.243666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98f49-68bb-4bec-83b8-400e470a87eb None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.244542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a98f49-68bb-4bec-83b8-400e470a87eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:09.244853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a98f49-68bb-4bec-83b8-400e470a87eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:09.246242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a98f49-68bb-4bec-83b8-400e470a87eb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:09.253946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 37/72] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:09.258468 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98f49-68bb-4bec-83b8-400e470a87eb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:09.263529 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7379f76-b2e6-49ac-836c-68873285ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:09.264190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7379f76-b2e6-49ac-836c-68873285ed5e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.264475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7379f76-b2e6-49ac-836c-68873285ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.264776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7379f76-b2e6-49ac-836c-68873285ed5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.332841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7379f76-b2e6-49ac-836c-68873285ed5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:09.334341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7379f76-b2e6-49ac-836c-68873285ed5e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:09.352457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98f49-68bb-4bec-83b8-400e470a87eb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:09.386709 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ef77b-6a27-493a-935d-9dc85dd418ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:09.387503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ef77b-6a27-493a-935d-9dc85dd418ad None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.387785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ef77b-6a27-493a-935d-9dc85dd418ad None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.388055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ef77b-6a27-493a-935d-9dc85dd418ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.398058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ef77b-6a27-493a-935d-9dc85dd418ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-854ef77b-6a27-493a-935d-9dc85dd418ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:09.406015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:09.406871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:09.407660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 38/73] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:09.408414 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352201-e525-4155-830b-88dced4e7e5e None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.408729 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352201-e525-4155-830b-88dced4e7e5e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.409000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352201-e525-4155-830b-88dced4e7e5e None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.410195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 39/74] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:09.411241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa02f5c-3757-4ad0-b00c-d0c3f6c6e157 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.411566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa02f5c-3757-4ad0-b00c-d0c3f6c6e157 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.411787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa02f5c-3757-4ad0-b00c-d0c3f6c6e157 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.413360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 40/75] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:09.413746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa02f5c-3757-4ad0-b00c-d0c3f6c6e157 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:09.432451 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 36/76] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:09.433327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7379f76-b2e6-49ac-836c-68873285ed5e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:09.514981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa02f5c-3757-4ad0-b00c-d0c3f6c6e157 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:09.531853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7379f76-b2e6-49ac-836c-68873285ed5e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:09.549753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451626af-1ec4-4948-82f6-45f986ff7c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:09.550636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451626af-1ec4-4948-82f6-45f986ff7c1b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.550912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451626af-1ec4-4948-82f6-45f986ff7c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.551186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451626af-1ec4-4948-82f6-45f986ff7c1b None admin] PATH_INFO: `/v3/users/c920286345e24cc09a50a6ef9210d7fc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.564386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79c6a6-0d63-4fc8-90c4-00d3af2f8464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:09.564572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451626af-1ec4-4948-82f6-45f986ff7c1b None admin] RBAC: Authorizing `identity:get_user(user_id=c920286345e24cc09a50a6ef9210d7fc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:09.565089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79c6a6-0d63-4fc8-90c4-00d3af2f8464 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.565373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79c6a6-0d63-4fc8-90c4-00d3af2f8464 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.565644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79c6a6-0d63-4fc8-90c4-00d3af2f8464 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.565999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451626af-1ec4-4948-82f6-45f986ff7c1b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:09.566398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79c6a6-0d63-4fc8-90c4-00d3af2f8464 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:09.566701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79c6a6-0d63-4fc8-90c4-00d3af2f8464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:09.567659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 41/77] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:09 2022] GET /identity/v3/users/c920286345e24cc09a50a6ef9210d7fc => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:09.568074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79c6a6-0d63-4fc8-90c4-00d3af2f8464 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:09.569812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d677a80-f326-49a0-9ac0-7aaba2870487 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.570094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d677a80-f326-49a0-9ac0-7aaba2870487 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.570368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d677a80-f326-49a0-9ac0-7aaba2870487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.576050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 37/78] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:09.577327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be5cec-78ee-4538-b3ef-e2f792ce50cf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.577616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be5cec-78ee-4538-b3ef-e2f792ce50cf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.577890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be5cec-78ee-4538-b3ef-e2f792ce50cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.586506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1d677a80-f326-49a0-9ac0-7aaba2870487 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:09.587669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1d677a80-f326-49a0-9ac0-7aaba2870487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:09.594049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-08be5cec-78ee-4538-b3ef-e2f792ce50cf None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:09.595191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-08be5cec-78ee-4538-b3ef-e2f792ce50cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:09.652629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 42/79] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:09.653916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d677a80-f326-49a0-9ac0-7aaba2870487 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:09.658479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 38/80] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:09.659316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be5cec-78ee-4538-b3ef-e2f792ce50cf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:09.748736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d677a80-f326-49a0-9ac0-7aaba2870487 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:09.758023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be5cec-78ee-4538-b3ef-e2f792ce50cf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:09.781725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350d082-c31e-4f7d-8664-9c91a4aa4dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:09.782619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350d082-c31e-4f7d-8664-9c91a4aa4dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.782941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350d082-c31e-4f7d-8664-9c91a4aa4dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.783198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350d082-c31e-4f7d-8664-9c91a4aa4dd3 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.784429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350d082-c31e-4f7d-8664-9c91a4aa4dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:09.785939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350d082-c31e-4f7d-8664-9c91a4aa4dd3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:09.787650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 43/81] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:09 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:09.788406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350d082-c31e-4f7d-8664-9c91a4aa4dd3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:09.791885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589eab5-3598-425f-abfe-d2ab14c0e2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:09.793320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589eab5-3598-425f-abfe-d2ab14c0e2da None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.793320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589eab5-3598-425f-abfe-d2ab14c0e2da None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.793320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589eab5-3598-425f-abfe-d2ab14c0e2da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.856972 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589eab5-3598-425f-abfe-d2ab14c0e2da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:09.858511 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589eab5-3598-425f-abfe-d2ab14c0e2da None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:09.892341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350d082-c31e-4f7d-8664-9c91a4aa4dd3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:09.926500 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a82444-e22a-4e8d-ab26-bb0ea80f68b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:09.927381 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a82444-e22a-4e8d-ab26-bb0ea80f68b6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:09.927657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a82444-e22a-4e8d-ab26-bb0ea80f68b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:09.927925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a82444-e22a-4e8d-ab26-bb0ea80f68b6 None admin] PATH_INFO: `/v3/users/c920286345e24cc09a50a6ef9210d7fc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:09.929176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a82444-e22a-4e8d-ab26-bb0ea80f68b6 None admin] RBAC: Authorizing `identity:get_user(user_id=c920286345e24cc09a50a6ef9210d7fc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:09.930639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a82444-e22a-4e8d-ab26-bb0ea80f68b6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:09.932661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 44/82] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:09 2022] GET /identity/v3/users/c920286345e24cc09a50a6ef9210d7fc => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:09.933182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a82444-e22a-4e8d-ab26-bb0ea80f68b6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:09.958621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 39/83] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:09.959452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589eab5-3598-425f-abfe-d2ab14c0e2da None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.035561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a82444-e22a-4e8d-ab26-bb0ea80f68b6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.056300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589eab5-3598-425f-abfe-d2ab14c0e2da None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.068698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce748b05-2f71-4b16-835e-4f3715d2b960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.069499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce748b05-2f71-4b16-835e-4f3715d2b960 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.069783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce748b05-2f71-4b16-835e-4f3715d2b960 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.070056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce748b05-2f71-4b16-835e-4f3715d2b960 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.079788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce748b05-2f71-4b16-835e-4f3715d2b960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ce748b05-2f71-4b16-835e-4f3715d2b960 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:10.087303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:10.088240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:10.094484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:10.095051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78c3d3-82e0-44b9-b22b-db0adc008342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.095051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c3d3-82e0-44b9-b22b-db0adc008342 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.095185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 45/84] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:10.095255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c3d3-82e0-44b9-b22b-db0adc008342 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.095255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c3d3-82e0-44b9-b22b-db0adc008342 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.095375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce748b05-2f71-4b16-835e-4f3715d2b960 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.095436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78c3d3-82e0-44b9-b22b-db0adc008342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:10.095436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78c3d3-82e0-44b9-b22b-db0adc008342 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:10.095436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 40/85] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:10.095436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78c3d3-82e0-44b9-b22b-db0adc008342 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.188105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce748b05-2f71-4b16-835e-4f3715d2b960 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.192515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78c3d3-82e0-44b9-b22b-db0adc008342 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.227812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e2706-a455-4963-ae83-01bc8eb0b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.228465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e2706-a455-4963-ae83-01bc8eb0b25c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.228915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f2249-2d4b-4d44-b91d-976bc60f1760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.229028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e2706-a455-4963-ae83-01bc8eb0b25c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.229349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e2706-a455-4963-ae83-01bc8eb0b25c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.229758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f2249-2d4b-4d44-b91d-976bc60f1760 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.230047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f2249-2d4b-4d44-b91d-976bc60f1760 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.230328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f2249-2d4b-4d44-b91d-976bc60f1760 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.231167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f2249-2d4b-4d44-b91d-976bc60f1760 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=7a48da3b90914e098e862c1aadf1620b, user.id=a74b9eac8d87492aad683f6da38fa8b9) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:10.231480 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f2249-2d4b-4d44-b91d-976bc60f1760 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:10.232954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f2249-2d4b-4d44-b91d-976bc60f1760 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:10.238171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 46/86] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:10 2022] GET /identity/v3/role_assignments?user.id=a74b9eac8d87492aad683f6da38fa8b9&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=e55ef11844a742f5889243f189bd59bc => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:10.238948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f2249-2d4b-4d44-b91d-976bc60f1760 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.306422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e2706-a455-4963-ae83-01bc8eb0b25c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:10.307957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e2706-a455-4963-ae83-01bc8eb0b25c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:10.345255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f2249-2d4b-4d44-b91d-976bc60f1760 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.378185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99602852-7427-43f0-892e-f1de9cee4e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.379083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99602852-7427-43f0-892e-f1de9cee4e6b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.379371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99602852-7427-43f0-892e-f1de9cee4e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.379650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99602852-7427-43f0-892e-f1de9cee4e6b None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.380986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99602852-7427-43f0-892e-f1de9cee4e6b None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:10.382482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99602852-7427-43f0-892e-f1de9cee4e6b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:10.389506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 47/87] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:10 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:10.390433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99602852-7427-43f0-892e-f1de9cee4e6b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.422107 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 41/88] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:10.422558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e2706-a455-4963-ae83-01bc8eb0b25c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.492795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99602852-7427-43f0-892e-f1de9cee4e6b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.519203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e2706-a455-4963-ae83-01bc8eb0b25c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.525741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324d153-020e-42bc-b5c7-f9812f69adfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.526431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324d153-020e-42bc-b5c7-f9812f69adfe None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.526713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324d153-020e-42bc-b5c7-f9812f69adfe None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.526984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324d153-020e-42bc-b5c7-f9812f69adfe None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.527692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324d153-020e-42bc-b5c7-f9812f69adfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:10.529160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324d153-020e-42bc-b5c7-f9812f69adfe None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:10.551873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d744b-b334-406a-b748-03c7e19e7633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.552731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d744b-b334-406a-b748-03c7e19e7633 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.553025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d744b-b334-406a-b748-03c7e19e7633 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.553295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d744b-b334-406a-b748-03c7e19e7633 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.554037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d744b-b334-406a-b748-03c7e19e7633 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:10.554342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d744b-b334-406a-b748-03c7e19e7633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:10.562878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 48/89] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:29:10 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:10.563045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d744b-b334-406a-b748-03c7e19e7633 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:10.563138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324d153-020e-42bc-b5c7-f9812f69adfe None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.565015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 42/90] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:10.565874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d744b-b334-406a-b748-03c7e19e7633 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.672180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d744b-b334-406a-b748-03c7e19e7633 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.675695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324d153-020e-42bc-b5c7-f9812f69adfe None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.707621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196fdd1-b71a-4c76-8e90-a1afc2e33246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.708516 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196fdd1-b71a-4c76-8e90-a1afc2e33246 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.708825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196fdd1-b71a-4c76-8e90-a1afc2e33246 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.709094 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196fdd1-b71a-4c76-8e90-a1afc2e33246 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.710327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196fdd1-b71a-4c76-8e90-a1afc2e33246 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:10.711547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1553a7-0f7a-4efe-a5db-5fc5c52859b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.711775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196fdd1-b71a-4c76-8e90-a1afc2e33246 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:10.712363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1553a7-0f7a-4efe-a5db-5fc5c52859b5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.712742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1553a7-0f7a-4efe-a5db-5fc5c52859b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.713384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1553a7-0f7a-4efe-a5db-5fc5c52859b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.714002 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 43/91] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:10 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:10.714465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1553a7-0f7a-4efe-a5db-5fc5c52859b5 None admin] RBAC: Adding query filter params (role.id=3385b41b990e4c6b88d77260a7822c7e, scope.project.id=7a48da3b90914e098e862c1aadf1620b, user.id=c920286345e24cc09a50a6ef9210d7fc) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:10.714847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1553a7-0f7a-4efe-a5db-5fc5c52859b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:10.714955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196fdd1-b71a-4c76-8e90-a1afc2e33246 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.716467 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1553a7-0f7a-4efe-a5db-5fc5c52859b5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:10.723016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 49/92] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:10 2022] GET /identity/v3/role_assignments?user.id=c920286345e24cc09a50a6ef9210d7fc&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:10.724061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1553a7-0f7a-4efe-a5db-5fc5c52859b5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.832254 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196fdd1-b71a-4c76-8e90-a1afc2e33246 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.836213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1553a7-0f7a-4efe-a5db-5fc5c52859b5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:10.869403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeda0b-c005-4c3a-8347-07824d6e87e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.870360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeda0b-c005-4c3a-8347-07824d6e87e0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.870594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeda0b-c005-4c3a-8347-07824d6e87e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.870858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeda0b-c005-4c3a-8347-07824d6e87e0 None admin] PATH_INFO: `/v3/users/c920286345e24cc09a50a6ef9210d7fc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.872128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeda0b-c005-4c3a-8347-07824d6e87e0 None admin] RBAC: Authorizing `identity:get_user(user_id=c920286345e24cc09a50a6ef9210d7fc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:10.873817 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeda0b-c005-4c3a-8347-07824d6e87e0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:10.874385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435e78-c35a-4671-b775-d11ef8df66cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:10.875124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435e78-c35a-4671-b775-d11ef8df66cc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:10.875409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435e78-c35a-4671-b775-d11ef8df66cc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:10.875683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435e78-c35a-4671-b775-d11ef8df66cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:10.876567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435e78-c35a-4671-b775-d11ef8df66cc None admin] RBAC: Adding query filter params (role.id=ec08a86158a44ee2b82951b44b228df2, scope.project.id=7a48da3b90914e098e862c1aadf1620b, user.id=c920286345e24cc09a50a6ef9210d7fc) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:10.876882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435e78-c35a-4671-b775-d11ef8df66cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:10.878464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21435e78-c35a-4671-b775-d11ef8df66cc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:10.880290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 44/93] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:10 2022] GET /identity/v3/users/c920286345e24cc09a50a6ef9210d7fc => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:10.881695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeda0b-c005-4c3a-8347-07824d6e87e0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:10.886484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 50/94] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:10 2022] GET /identity/v3/role_assignments?user.id=c920286345e24cc09a50a6ef9210d7fc&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=ec08a86158a44ee2b82951b44b228df2 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:10.998220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeda0b-c005-4c3a-8347-07824d6e87e0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:11.034048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5dceed-7e78-4bdf-b4ef-84c2e0b997b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:11.034724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5dceed-7e78-4bdf-b4ef-84c2e0b997b9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:11.035010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5dceed-7e78-4bdf-b4ef-84c2e0b997b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:11.035286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5dceed-7e78-4bdf-b4ef-84c2e0b997b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:11.110912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5dceed-7e78-4bdf-b4ef-84c2e0b997b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:11.112308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5dceed-7e78-4bdf-b4ef-84c2e0b997b9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:11.220938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 45/95] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:11.226629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435e78-c35a-4671-b775-d11ef8df66cc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:11.329236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21435e78-c35a-4671-b775-d11ef8df66cc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:11.367009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c137f01-b002-45be-9ec3-0d0642ef5bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:11.368293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c137f01-b002-45be-9ec3-0d0642ef5bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:11.368788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c137f01-b002-45be-9ec3-0d0642ef5bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:11.369350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c137f01-b002-45be-9ec3-0d0642ef5bd6 None admin] PATH_INFO: `/v3/projects/3cca2b4862f24dbb8647acba4b570f23` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:11.373304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c137f01-b002-45be-9ec3-0d0642ef5bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=3cca2b4862f24dbb8647acba4b570f23)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:11.374803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c137f01-b002-45be-9ec3-0d0642ef5bd6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:11.376789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 51/96] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:11 2022] GET /identity/v3/projects/3cca2b4862f24dbb8647acba4b570f23 => generated 312 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:11.382906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5dceed-7e78-4bdf-b4ef-84c2e0b997b9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:11.577363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5dceed-7e78-4bdf-b4ef-84c2e0b997b9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:11.622219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1e0f6-05d6-448d-a628-2a8ef8b559c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:11.623743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1e0f6-05d6-448d-a628-2a8ef8b559c0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:11.624236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1e0f6-05d6-448d-a628-2a8ef8b559c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:11.624726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1e0f6-05d6-448d-a628-2a8ef8b559c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:11.625790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1e0f6-05d6-448d-a628-2a8ef8b559c0 None admin] RBAC: Adding query filter params (role.id=3385b41b990e4c6b88d77260a7822c7e, scope.project.id=3cca2b4862f24dbb8647acba4b570f23, user.id=c920286345e24cc09a50a6ef9210d7fc) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:11.626297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1e0f6-05d6-448d-a628-2a8ef8b559c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:11.627995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1e0f6-05d6-448d-a628-2a8ef8b559c0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:11.640632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 46/97] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:11 2022] GET /identity/v3/role_assignments?user.id=c920286345e24cc09a50a6ef9210d7fc&scope.project.id=3cca2b4862f24dbb8647acba4b570f23&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:13.207704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e9072-b8e5-4f1b-a0a8-2e5110e3a849 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:13.208283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e9072-b8e5-4f1b-a0a8-2e5110e3a849 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:13.208283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e9072-b8e5-4f1b-a0a8-2e5110e3a849 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:13.209670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 52/98] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:13.217049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9ef9d-be23-4422-a999-c2a415a1f239 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:13.217049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9ef9d-be23-4422-a999-c2a415a1f239 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:13.217049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9ef9d-be23-4422-a999-c2a415a1f239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:13.237802 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f9e9ef9d-be23-4422-a999-c2a415a1f239 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:13.239334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f9e9ef9d-be23-4422-a999-c2a415a1f239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:13.323290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 47/99] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:13.353007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e9072-b8e5-4f1b-a0a8-2e5110e3a849 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:13.459982 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e9072-b8e5-4f1b-a0a8-2e5110e3a849 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:13.493684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfa2a6-6873-4f67-ab50-cb287a147e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:13.494608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfa2a6-6873-4f67-ab50-cb287a147e92 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:13.494894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfa2a6-6873-4f67-ab50-cb287a147e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:13.495225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfa2a6-6873-4f67-ab50-cb287a147e92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:13.505692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bfa2a6-6873-4f67-ab50-cb287a147e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-b3bfa2a6-6873-4f67-ab50-cb287a147e92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:13.513561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:13.514462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:13.515495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 53/100] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:29:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:13.520978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9ef9d-be23-4422-a999-c2a415a1f239 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:13.677448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9ef9d-be23-4422-a999-c2a415a1f239 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:13.722684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e787efe-54a4-48dd-9b02-3b4d3a622de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:13.723288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e787efe-54a4-48dd-9b02-3b4d3a622de1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:13.723568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e787efe-54a4-48dd-9b02-3b4d3a622de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:13.723838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e787efe-54a4-48dd-9b02-3b4d3a622de1 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:13.724627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e787efe-54a4-48dd-9b02-3b4d3a622de1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:13.724932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e787efe-54a4-48dd-9b02-3b4d3a622de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:13.726358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e787efe-54a4-48dd-9b02-3b4d3a622de1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:13.739777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 48/101] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:13.746921 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfa2a6-6873-4f67-ab50-cb287a147e92 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:13.869380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfa2a6-6873-4f67-ab50-cb287a147e92 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:13.903435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197a1a0-c1fd-46ee-9d8d-250806c26d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:13.904341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197a1a0-c1fd-46ee-9d8d-250806c26d43 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:13.904679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197a1a0-c1fd-46ee-9d8d-250806c26d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:13.904960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197a1a0-c1fd-46ee-9d8d-250806c26d43 None admin] PATH_INFO: `/v3/users/a74b9eac8d87492aad683f6da38fa8b9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:13.906221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197a1a0-c1fd-46ee-9d8d-250806c26d43 None admin] RBAC: Authorizing `identity:get_user(user_id=a74b9eac8d87492aad683f6da38fa8b9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:13.907686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197a1a0-c1fd-46ee-9d8d-250806c26d43 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:13.909817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 54/102] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:13 2022] GET /identity/v3/users/a74b9eac8d87492aad683f6da38fa8b9 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:13.916771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e787efe-54a4-48dd-9b02-3b4d3a622de1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:14.067533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e787efe-54a4-48dd-9b02-3b4d3a622de1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:14.083949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a29d6e-8d81-4854-995a-1e3f333503d8 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:14.084246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a29d6e-8d81-4854-995a-1e3f333503d8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:14.084712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a29d6e-8d81-4854-995a-1e3f333503d8 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:14.086171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 55/103] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:14.092551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e8e8-1e01-4568-99fe-ed7990e28141 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:14.092925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e8e8-1e01-4568-99fe-ed7990e28141 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:14.093280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3e8e8-1e01-4568-99fe-ed7990e28141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:14.113202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc07bc-0947-4a49-a017-a34796be9745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:14.114089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc07bc-0947-4a49-a017-a34796be9745 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:14.114371 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc07bc-0947-4a49-a017-a34796be9745 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:14.114645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc07bc-0947-4a49-a017-a34796be9745 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:14.115877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc07bc-0947-4a49-a017-a34796be9745 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:14.117351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc07bc-0947-4a49-a017-a34796be9745 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:14.119058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 49/104] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:13 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:14.125842 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc07bc-0947-4a49-a017-a34796be9745 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:14.126003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d5a3e8e8-1e01-4568-99fe-ed7990e28141 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:14.127191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d5a3e8e8-1e01-4568-99fe-ed7990e28141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:14.286931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 56/105] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:14.297664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce0756-3762-4230-ad99-5fbef775eb9a None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:14.298267 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce0756-3762-4230-ad99-5fbef775eb9a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:14.298769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce0756-3762-4230-ad99-5fbef775eb9a None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:14.300315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 57/106] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:29:14.305422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d29e69-14ba-4bb6-9953-3549e414b163 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:14.305927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d29e69-14ba-4bb6-9953-3549e414b163 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:14.306408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d29e69-14ba-4bb6-9953-3549e414b163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:14.310555 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc07bc-0947-4a49-a017-a34796be9745 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:14.342902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e3d29e69-14ba-4bb6-9953-3549e414b163 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:14.345021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e3d29e69-14ba-4bb6-9953-3549e414b163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:14.354764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f84080-3305-451c-a687-ae359c46375b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:14.356034 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84080-3305-451c-a687-ae359c46375b None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:14.356651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84080-3305-451c-a687-ae359c46375b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:14.357133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84080-3305-451c-a687-ae359c46375b None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b/users/a74b9eac8d87492aad683f6da38fa8b9/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:14.361274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f84080-3305-451c-a687-ae359c46375b None admin] RBAC: Authorizing `identity:create_grant(project_id=7a48da3b90914e098e862c1aadf1620b, user_id=a74b9eac8d87492aad683f6da38fa8b9, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:14.362966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f84080-3305-451c-a687-ae359c46375b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:14.387910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 50/107] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:29:14 2022] PUT /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b/users/a74b9eac8d87492aad683f6da38fa8b9/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:29:14.389296 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f84080-3305-451c-a687-ae359c46375b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:14.452084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 58/108] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:14.506843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d29e69-14ba-4bb6-9953-3549e414b163 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:14.525070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f84080-3305-451c-a687-ae359c46375b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:14.564604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c131f09-4dd4-40b3-8b11-b8ffe9b599ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:14.564604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c131f09-4dd4-40b3-8b11-b8ffe9b599ed None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:14.564604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c131f09-4dd4-40b3-8b11-b8ffe9b599ed None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:14.564604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c131f09-4dd4-40b3-8b11-b8ffe9b599ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:14.586074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c131f09-4dd4-40b3-8b11-b8ffe9b599ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0c131f09-4dd4-40b3-8b11-b8ffe9b599ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:14.602977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:14.604761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:14.605777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:14.607445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 51/109] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:14.613234 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c131f09-4dd4-40b3-8b11-b8ffe9b599ed None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:14.700510 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d29e69-14ba-4bb6-9953-3549e414b163 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:14.790913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c78c6-6e21-4045-8da7-1d9bd698feab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:14.792241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c78c6-6e21-4045-8da7-1d9bd698feab None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:14.792548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c78c6-6e21-4045-8da7-1d9bd698feab None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:14.792825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c78c6-6e21-4045-8da7-1d9bd698feab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:14.807028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c78c6-6e21-4045-8da7-1d9bd698feab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f24c78c6-6e21-4045-8da7-1d9bd698feab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:14.815496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:14.816371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:14.817183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 59/110] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:14.817814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c131f09-4dd4-40b3-8b11-b8ffe9b599ed None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:14.825161 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c78c6-6e21-4045-8da7-1d9bd698feab None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:14.867830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e2b88-7eac-46d3-ac6a-98f4463aff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:14.868647 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e2b88-7eac-46d3-ac6a-98f4463aff5e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:14.868938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e2b88-7eac-46d3-ac6a-98f4463aff5e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:14.869305 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e2b88-7eac-46d3-ac6a-98f4463aff5e None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:14.870425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e2b88-7eac-46d3-ac6a-98f4463aff5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:14.870735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e2b88-7eac-46d3-ac6a-98f4463aff5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:14.872542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e2b88-7eac-46d3-ac6a-98f4463aff5e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:14.883938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 52/111] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:14.889825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e2b88-7eac-46d3-ac6a-98f4463aff5e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:14.953428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c78c6-6e21-4045-8da7-1d9bd698feab None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:14.997165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8111b-8754-41d5-80d5-539aa7355fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:14.998367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8111b-8754-41d5-80d5-539aa7355fed None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:14.998792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8111b-8754-41d5-80d5-539aa7355fed None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:14.999205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8111b-8754-41d5-80d5-539aa7355fed None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:14.999785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e2b88-7eac-46d3-ac6a-98f4463aff5e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.000274 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8111b-8754-41d5-80d5-539aa7355fed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:15.000753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8111b-8754-41d5-80d5-539aa7355fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:15.003246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8111b-8754-41d5-80d5-539aa7355fed None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:15.014173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 60/112] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:15.015994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196705f-1227-4cc0-b9f6-416be5dc82d0 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.016631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196705f-1227-4cc0-b9f6-416be5dc82d0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.017065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196705f-1227-4cc0-b9f6-416be5dc82d0 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.018795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 61/113] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:15.020011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967c87-e729-4775-aa1c-e873d84c086a None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.020891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967c87-e729-4775-aa1c-e873d84c086a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.021311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967c87-e729-4775-aa1c-e873d84c086a None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.022979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 62/114] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:15.023974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967c87-e729-4775-aa1c-e873d84c086a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:15.045645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c07a33-8d5b-4ffd-a50a-69b6bddf6a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.046779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c07a33-8d5b-4ffd-a50a-69b6bddf6a63 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.047221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c07a33-8d5b-4ffd-a50a-69b6bddf6a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.048063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c07a33-8d5b-4ffd-a50a-69b6bddf6a63 None admin] PATH_INFO: `/v3/users/c920286345e24cc09a50a6ef9210d7fc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.049776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c07a33-8d5b-4ffd-a50a-69b6bddf6a63 None admin] RBAC: Authorizing `identity:get_user(user_id=c920286345e24cc09a50a6ef9210d7fc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:15.051901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c07a33-8d5b-4ffd-a50a-69b6bddf6a63 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:15.054514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 53/115] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:14 2022] GET /identity/v3/users/c920286345e24cc09a50a6ef9210d7fc => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:15.056120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c125b6-a11d-4214-ae1a-3b0d697f8eea None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.056863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c125b6-a11d-4214-ae1a-3b0d697f8eea None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.057300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c125b6-a11d-4214-ae1a-3b0d697f8eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.079515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-41c125b6-a11d-4214-ae1a-3b0d697f8eea None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:15.081008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-41c125b6-a11d-4214-ae1a-3b0d697f8eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:15.128589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967c87-e729-4775-aa1c-e873d84c086a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.151823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 54/116] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:15.154454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b685f4-1064-4040-9b68-e7ca6f74b07f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.154750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b685f4-1064-4040-9b68-e7ca6f74b07f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.155094 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b685f4-1064-4040-9b68-e7ca6f74b07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.161982 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a3fdd-e95a-4199-9f38-42b662485841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.163141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a3fdd-e95a-4199-9f38-42b662485841 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.163615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a3fdd-e95a-4199-9f38-42b662485841 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.164114 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a3fdd-e95a-4199-9f38-42b662485841 None admin] PATH_INFO: `/v3/users/c920286345e24cc09a50a6ef9210d7fc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.165559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a3fdd-e95a-4199-9f38-42b662485841 None admin] RBAC: Authorizing `identity:get_user(user_id=c920286345e24cc09a50a6ef9210d7fc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:15.167275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a3fdd-e95a-4199-9f38-42b662485841 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:15.169294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 63/117] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:15 2022] GET /identity/v3/users/c920286345e24cc09a50a6ef9210d7fc => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:15.170242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a3fdd-e95a-4199-9f38-42b662485841 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:15.175367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b0b685f4-1064-4040-9b68-e7ca6f74b07f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:15.176598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b0b685f4-1064-4040-9b68-e7ca6f74b07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:15.239599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 55/118] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:15.244777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf08c4-2445-436b-b444-40a842546ae8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.244777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf08c4-2445-436b-b444-40a842546ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.244777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf08c4-2445-436b-b444-40a842546ae8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.244777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 56/119] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:15.244777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf08c4-2445-436b-b444-40a842546ae8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:15.277873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a3fdd-e95a-4199-9f38-42b662485841 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.311385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ebb5e-3e3d-4d65-bc54-9f404bfe8f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.312283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ebb5e-3e3d-4d65-bc54-9f404bfe8f47 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.312584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ebb5e-3e3d-4d65-bc54-9f404bfe8f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.312852 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ebb5e-3e3d-4d65-bc54-9f404bfe8f47 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.314015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ebb5e-3e3d-4d65-bc54-9f404bfe8f47 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:15.315451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ebb5e-3e3d-4d65-bc54-9f404bfe8f47 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:15.317337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 64/120] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:15 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:15.317918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ebb5e-3e3d-4d65-bc54-9f404bfe8f47 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:15.351067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf08c4-2445-436b-b444-40a842546ae8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.384938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ea322-a3ec-4bef-9b43-e1201b33ca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.385710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ea322-a3ec-4bef-9b43-e1201b33ca51 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.385989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ea322-a3ec-4bef-9b43-e1201b33ca51 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.386249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ea322-a3ec-4bef-9b43-e1201b33ca51 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.387429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ea322-a3ec-4bef-9b43-e1201b33ca51 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:15.388905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ea322-a3ec-4bef-9b43-e1201b33ca51 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:15.395613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 57/121] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:15 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:15.395613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b368161-d04a-4d0b-944f-1e19664ad8cc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.395613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b368161-d04a-4d0b-944f-1e19664ad8cc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.395613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b368161-d04a-4d0b-944f-1e19664ad8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.410146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2b368161-d04a-4d0b-944f-1e19664ad8cc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:15.411421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2b368161-d04a-4d0b-944f-1e19664ad8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:15.420947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ebb5e-3e3d-4d65-bc54-9f404bfe8f47 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.453439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb03a69-4ef9-4384-8999-6c271e53f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.454243 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03a69-4ef9-4384-8999-6c271e53f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.454527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03a69-4ef9-4384-8999-6c271e53f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.454799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03a69-4ef9-4384-8999-6c271e53f8f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.465225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb03a69-4ef9-4384-8999-6c271e53f8f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-aeb03a69-4ef9-4384-8999-6c271e53f8f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:15.473319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:15.474176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:15.474931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 65/122] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:15.475548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb03a69-4ef9-4384-8999-6c271e53f8f5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:15.478242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 58/123] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:15.478991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b368161-d04a-4d0b-944f-1e19664ad8cc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:15.574820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b368161-d04a-4d0b-944f-1e19664ad8cc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.583685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb03a69-4ef9-4384-8999-6c271e53f8f5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.612648 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee30e9-6cf3-4466-bb57-3cdb36b35a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.613543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee30e9-6cf3-4466-bb57-3cdb36b35a4a None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.613821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee30e9-6cf3-4466-bb57-3cdb36b35a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.614092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee30e9-6cf3-4466-bb57-3cdb36b35a4a None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b/users/c920286345e24cc09a50a6ef9210d7fc/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.616040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee30e9-6cf3-4466-bb57-3cdb36b35a4a None admin] RBAC: Authorizing `identity:create_grant(project_id=7a48da3b90914e098e862c1aadf1620b, user_id=c920286345e24cc09a50a6ef9210d7fc, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:15.617487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee30e9-6cf3-4466-bb57-3cdb36b35a4a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:15.622080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a49618-ac11-42f2-8ef5-57b1f5645187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.622922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49618-ac11-42f2-8ef5-57b1f5645187 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.623200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49618-ac11-42f2-8ef5-57b1f5645187 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.623514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49618-ac11-42f2-8ef5-57b1f5645187 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.624930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a49618-ac11-42f2-8ef5-57b1f5645187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:15.626510 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a49618-ac11-42f2-8ef5-57b1f5645187 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:15.628671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 66/124] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:15.629557 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a49618-ac11-42f2-8ef5-57b1f5645187 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:15.638287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 59/125] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:29:15 2022] PUT /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b/users/c920286345e24cc09a50a6ef9210d7fc/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:15.639033 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee30e9-6cf3-4466-bb57-3cdb36b35a4a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:15.762962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee30e9-6cf3-4466-bb57-3cdb36b35a4a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.771346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a49618-ac11-42f2-8ef5-57b1f5645187 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.799324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0cc20-bda2-4a6d-9367-825edf5378cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.800073 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0cc20-bda2-4a6d-9367-825edf5378cc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.800357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0cc20-bda2-4a6d-9367-825edf5378cc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.800714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0cc20-bda2-4a6d-9367-825edf5378cc None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.801484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0cc20-bda2-4a6d-9367-825edf5378cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:15.801802 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0cc20-bda2-4a6d-9367-825edf5378cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:15.803337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0cc20-bda2-4a6d-9367-825edf5378cc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:15.813138 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 60/126] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:15.813138 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0cc20-bda2-4a6d-9367-825edf5378cc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:15.821914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082e64-df2c-4f16-8fd6-a48f2a5ec0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.823249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082e64-df2c-4f16-8fd6-a48f2a5ec0dd None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.823529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082e64-df2c-4f16-8fd6-a48f2a5ec0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.823858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082e64-df2c-4f16-8fd6-a48f2a5ec0dd None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b/users/c920286345e24cc09a50a6ef9210d7fc/roles/ec08a86158a44ee2b82951b44b228df2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.826286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b082e64-df2c-4f16-8fd6-a48f2a5ec0dd None admin] RBAC: Authorizing `identity:create_grant(project_id=7a48da3b90914e098e862c1aadf1620b, user_id=c920286345e24cc09a50a6ef9210d7fc, role_id=ec08a86158a44ee2b82951b44b228df2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:15.827958 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b082e64-df2c-4f16-8fd6-a48f2a5ec0dd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:15.844800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 67/127] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:29:15 2022] PUT /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b/users/c920286345e24cc09a50a6ef9210d7fc/roles/ec08a86158a44ee2b82951b44b228df2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:15.845622 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082e64-df2c-4f16-8fd6-a48f2a5ec0dd None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:15.932186 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0cc20-bda2-4a6d-9367-825edf5378cc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.954003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082e64-df2c-4f16-8fd6-a48f2a5ec0dd None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:15.967929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60a376-a926-432b-a038-4cb04534c4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.969632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60a376-a926-432b-a038-4cb04534c4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.970106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60a376-a926-432b-a038-4cb04534c4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.970565 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60a376-a926-432b-a038-4cb04534c4a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.981976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60a376-a926-432b-a038-4cb04534c4a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:15.987731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91590646-d5c8-453a-8373-d0696c2ad3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:15.988407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590646-d5c8-453a-8373-d0696c2ad3a0 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:15.988717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590646-d5c8-453a-8373-d0696c2ad3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:15.988989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590646-d5c8-453a-8373-d0696c2ad3a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:15.989713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91590646-d5c8-453a-8373-d0696c2ad3a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:15.991158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91590646-d5c8-453a-8373-d0696c2ad3a0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4b60a376-a926-432b-a038-4cb04534c4a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:15.992075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:15.993019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:15.993966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:15.996761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 61/128] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:15.997825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60a376-a926-432b-a038-4cb04534c4a8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:16.022558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 68/129] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:29:15 2022] POST /identity/v3/projects => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:16.023903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91590646-d5c8-453a-8373-d0696c2ad3a0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:16.141867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91590646-d5c8-453a-8373-d0696c2ad3a0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:16.183843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30feb19-a75c-48f4-9211-5f8ce0c07f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:16.185127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30feb19-a75c-48f4-9211-5f8ce0c07f62 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:16.185478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30feb19-a75c-48f4-9211-5f8ce0c07f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:16.185816 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30feb19-a75c-48f4-9211-5f8ce0c07f62 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:16.186792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30feb19-a75c-48f4-9211-5f8ce0c07f62 None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:16.187162 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30feb19-a75c-48f4-9211-5f8ce0c07f62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:16.189221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30feb19-a75c-48f4-9211-5f8ce0c07f62 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:16.190527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60a376-a926-432b-a038-4cb04534c4a8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:16.200540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 69/130] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:16.206422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30feb19-a75c-48f4-9211-5f8ce0c07f62 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:16.237238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c5843-f6ca-45c9-a689-a88edc52f850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:16.238149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c5843-f6ca-45c9-a689-a88edc52f850 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:16.238434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c5843-f6ca-45c9-a689-a88edc52f850 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:16.238710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c5843-f6ca-45c9-a689-a88edc52f850 None admin] PATH_INFO: `/v3/users/e4c0b3cc764c4805a21f8d15a56d137f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:16.263487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c5843-f6ca-45c9-a689-a88edc52f850 None admin] RBAC: Authorizing `identity:get_user(user_id=e4c0b3cc764c4805a21f8d15a56d137f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:16.265034 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c5843-f6ca-45c9-a689-a88edc52f850 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:16.268624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 62/131] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:15 2022] GET /identity/v3/users/e4c0b3cc764c4805a21f8d15a56d137f => generated 289 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:16.274323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c5843-f6ca-45c9-a689-a88edc52f850 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:16.314243 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30feb19-a75c-48f4-9211-5f8ce0c07f62 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:16.349509 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27a1b-4cb4-41af-85af-5d8eb0d706a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:16.350434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27a1b-4cb4-41af-85af-5d8eb0d706a9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:16.350717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27a1b-4cb4-41af-85af-5d8eb0d706a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:16.350989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27a1b-4cb4-41af-85af-5d8eb0d706a9 None admin] PATH_INFO: `/v3/users/c920286345e24cc09a50a6ef9210d7fc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:16.352184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc27a1b-4cb4-41af-85af-5d8eb0d706a9 None admin] RBAC: Authorizing `identity:get_user(user_id=c920286345e24cc09a50a6ef9210d7fc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:16.353742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc27a1b-4cb4-41af-85af-5d8eb0d706a9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:16.355387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 70/132] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:16 2022] GET /identity/v3/users/c920286345e24cc09a50a6ef9210d7fc => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:16.366327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27a1b-4cb4-41af-85af-5d8eb0d706a9 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:16.414312 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c5843-f6ca-45c9-a689-a88edc52f850 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:16.450663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2e59-c498-4123-974b-e89097b39eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:16.451243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2e59-c498-4123-974b-e89097b39eff None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:16.451526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2e59-c498-4123-974b-e89097b39eff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:16.451797 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2e59-c498-4123-974b-e89097b39eff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:16.523763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd2e59-c498-4123-974b-e89097b39eff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:16.525259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd2e59-c498-4123-974b-e89097b39eff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:16.558410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27a1b-4cb4-41af-85af-5d8eb0d706a9 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:16.613222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09d527-d03a-4ad1-93d8-290a078c68ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:16.614461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09d527-d03a-4ad1-93d8-290a078c68ab None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:16.614965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09d527-d03a-4ad1-93d8-290a078c68ab None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:16.615587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09d527-d03a-4ad1-93d8-290a078c68ab None admin] PATH_INFO: `/v3/projects/3cca2b4862f24dbb8647acba4b570f23` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:16.617062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09d527-d03a-4ad1-93d8-290a078c68ab None admin] RBAC: Authorizing `identity:get_project(project_id=3cca2b4862f24dbb8647acba4b570f23)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:16.619523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09d527-d03a-4ad1-93d8-290a078c68ab None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:16.621590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 63/133] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:16.623264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 71/134] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:16 2022] GET /identity/v3/projects/3cca2b4862f24dbb8647acba4b570f23 => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:16.627235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2e59-c498-4123-974b-e89097b39eff None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:16.627837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09d527-d03a-4ad1-93d8-290a078c68ab None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:16.721982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2e59-c498-4123-974b-e89097b39eff None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:16.761379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955608b-a4b1-4389-bfce-4d615ba87ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:16.762257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955608b-a4b1-4389-bfce-4d615ba87ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:16.762535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955608b-a4b1-4389-bfce-4d615ba87ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:16.762804 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955608b-a4b1-4389-bfce-4d615ba87ea4 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:16.764120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955608b-a4b1-4389-bfce-4d615ba87ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:16.766643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955608b-a4b1-4389-bfce-4d615ba87ea4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:16.768356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 64/135] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:16 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:16.775509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955608b-a4b1-4389-bfce-4d615ba87ea4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:16.785724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09d527-d03a-4ad1-93d8-290a078c68ab None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:16.850240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d96de-4fd3-4626-b225-71550751d225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:16.851162 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d96de-4fd3-4626-b225-71550751d225 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:16.851444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d96de-4fd3-4626-b225-71550751d225 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:16.851717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d96de-4fd3-4626-b225-71550751d225 None admin] PATH_INFO: `/v3/projects/3cca2b4862f24dbb8647acba4b570f23/users/c920286345e24cc09a50a6ef9210d7fc/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:16.858347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d96de-4fd3-4626-b225-71550751d225 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cca2b4862f24dbb8647acba4b570f23, user_id=c920286345e24cc09a50a6ef9210d7fc, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:16.859842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d96de-4fd3-4626-b225-71550751d225 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:16.883015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 72/136] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:29:16 2022] PUT /identity/v3/projects/3cca2b4862f24dbb8647acba4b570f23/users/c920286345e24cc09a50a6ef9210d7fc/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:29:16.945484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955608b-a4b1-4389-bfce-4d615ba87ea4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:16.990358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6243f-ea32-4916-95eb-72fb96a27306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:16.990939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6243f-ea32-4916-95eb-72fb96a27306 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:16.991220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6243f-ea32-4916-95eb-72fb96a27306 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:16.991540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6243f-ea32-4916-95eb-72fb96a27306 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:16.992411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6243f-ea32-4916-95eb-72fb96a27306 None admin] RBAC: Adding query filter params (role.id=c6d29efc5be34cc097e61371c95cd446, scope.project.id=7a48da3b90914e098e862c1aadf1620b, user.id=e4c0b3cc764c4805a21f8d15a56d137f) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:16.992792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6243f-ea32-4916-95eb-72fb96a27306 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:16.994315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6243f-ea32-4916-95eb-72fb96a27306 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:17.000399 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 65/137] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:16 2022] GET /identity/v3/role_assignments?user.id=e4c0b3cc764c4805a21f8d15a56d137f&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=c6d29efc5be34cc097e61371c95cd446 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:17.594433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d94a5e-cd18-4898-b4bb-458485627d40 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:17.594983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d94a5e-cd18-4898-b4bb-458485627d40 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:17.594983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d94a5e-cd18-4898-b4bb-458485627d40 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:17.596358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 73/138] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:17.607844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1fb37-c5a8-4aee-a011-15c578ef28d7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:17.608366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1fb37-c5a8-4aee-a011-15c578ef28d7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:17.608366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1fb37-c5a8-4aee-a011-15c578ef28d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:17.636929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4fa1fb37-c5a8-4aee-a011-15c578ef28d7 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:17.638200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4fa1fb37-c5a8-4aee-a011-15c578ef28d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:17.729742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 66/139] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:17.765087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d94a5e-cd18-4898-b4bb-458485627d40 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:17.875956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d94a5e-cd18-4898-b4bb-458485627d40 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:17.915496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04487d-e72a-48d1-8dce-23b97e355d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:17.916358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04487d-e72a-48d1-8dce-23b97e355d8b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:17.916665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04487d-e72a-48d1-8dce-23b97e355d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:17.916937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04487d-e72a-48d1-8dce-23b97e355d8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:17.933405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04487d-e72a-48d1-8dce-23b97e355d8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-4d04487d-e72a-48d1-8dce-23b97e355d8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:17.941078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:17.941924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:17.942665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 74/140] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:29:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:17.949724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1fb37-c5a8-4aee-a011-15c578ef28d7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:18.062991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1fb37-c5a8-4aee-a011-15c578ef28d7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:18.110314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebce77-bbf8-41fa-8b5d-5ad2b0584c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:18.111006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebce77-bbf8-41fa-8b5d-5ad2b0584c3c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:18.111288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebce77-bbf8-41fa-8b5d-5ad2b0584c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:18.111563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebce77-bbf8-41fa-8b5d-5ad2b0584c3c None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:18.112328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebce77-bbf8-41fa-8b5d-5ad2b0584c3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:18.112658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebce77-bbf8-41fa-8b5d-5ad2b0584c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:18.114067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebce77-bbf8-41fa-8b5d-5ad2b0584c3c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:18.128044 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 67/141] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:18.133674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04487d-e72a-48d1-8dce-23b97e355d8b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:18.237005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04487d-e72a-48d1-8dce-23b97e355d8b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:18.275827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337d1be-64bc-4455-b8af-bb0f196513fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:18.276908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337d1be-64bc-4455-b8af-bb0f196513fe None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:18.277191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337d1be-64bc-4455-b8af-bb0f196513fe None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:18.277465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337d1be-64bc-4455-b8af-bb0f196513fe None admin] PATH_INFO: `/v3/users/a74b9eac8d87492aad683f6da38fa8b9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:18.278692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337d1be-64bc-4455-b8af-bb0f196513fe None admin] RBAC: Authorizing `identity:get_user(user_id=a74b9eac8d87492aad683f6da38fa8b9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:18.280175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337d1be-64bc-4455-b8af-bb0f196513fe None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:18.281828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 75/142] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:18 2022] GET /identity/v3/users/a74b9eac8d87492aad683f6da38fa8b9 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:18.287456 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebce77-bbf8-41fa-8b5d-5ad2b0584c3c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:18.416285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebce77-bbf8-41fa-8b5d-5ad2b0584c3c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:18.459561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014638cd-ec47-4dda-90a5-c889e255a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:18.460653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014638cd-ec47-4dda-90a5-c889e255a390 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:18.461146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014638cd-ec47-4dda-90a5-c889e255a390 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:18.461631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014638cd-ec47-4dda-90a5-c889e255a390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:18.527703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014638cd-ec47-4dda-90a5-c889e255a390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:18.529604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014638cd-ec47-4dda-90a5-c889e255a390 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:18.650119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 68/143] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:18.657633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337d1be-64bc-4455-b8af-bb0f196513fe None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:18.784328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337d1be-64bc-4455-b8af-bb0f196513fe None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:18.812428 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc100884-b9c5-4def-a1ac-8f8240075447 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:18.812783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc100884-b9c5-4def-a1ac-8f8240075447 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:18.813045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc100884-b9c5-4def-a1ac-8f8240075447 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:18.814707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 69/144] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:18.821443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a97a7-d85f-4d6b-aba8-84447d914e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:18.822375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a97a7-d85f-4d6b-aba8-84447d914e73 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:18.822660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a97a7-d85f-4d6b-aba8-84447d914e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:18.822934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a97a7-d85f-4d6b-aba8-84447d914e73 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:18.824148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a97a7-d85f-4d6b-aba8-84447d914e73 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:18.825632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a97a7-d85f-4d6b-aba8-84447d914e73 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:18.825856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744f8fc-9bf8-4aeb-ae2c-899b738b7e21 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:18.826119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744f8fc-9bf8-4aeb-ae2c-899b738b7e21 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:18.826401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744f8fc-9bf8-4aeb-ae2c-899b738b7e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:18.827318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 76/145] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:18 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:18.833947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a97a7-d85f-4d6b-aba8-84447d914e73 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:18.868503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2744f8fc-9bf8-4aeb-ae2c-899b738b7e21 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:18.869886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2744f8fc-9bf8-4aeb-ae2c-899b738b7e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:18.976878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a97a7-d85f-4d6b-aba8-84447d914e73 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:19.015562 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c2900-e55e-4d29-baaf-e94d2b8ff9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:19.016237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c2900-e55e-4d29-baaf-e94d2b8ff9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:19.016538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c2900-e55e-4d29-baaf-e94d2b8ff9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:19.016821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c2900-e55e-4d29-baaf-e94d2b8ff9e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:19.017661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c2900-e55e-4d29-baaf-e94d2b8ff9e9 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=7a48da3b90914e098e862c1aadf1620b, user.id=a74b9eac8d87492aad683f6da38fa8b9) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:19.017980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c2900-e55e-4d29-baaf-e94d2b8ff9e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:19.019416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c2900-e55e-4d29-baaf-e94d2b8ff9e9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:19.025618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 77/146] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:18 2022] GET /identity/v3/role_assignments?user.id=a74b9eac8d87492aad683f6da38fa8b9&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=e55ef11844a742f5889243f189bd59bc => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:19.034286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 70/147] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:19.076837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c2900-e55e-4d29-baaf-e94d2b8ff9e9 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:19.184366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c2900-e55e-4d29-baaf-e94d2b8ff9e9 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:19.223787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48b697-477c-44f5-8ec7-de8e91642ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:19.224645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48b697-477c-44f5-8ec7-de8e91642ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:19.224933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48b697-477c-44f5-8ec7-de8e91642ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:19.225235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48b697-477c-44f5-8ec7-de8e91642ee3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:19.240025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48b697-477c-44f5-8ec7-de8e91642ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-3d48b697-477c-44f5-8ec7-de8e91642ee3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:19.248009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:19.248948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:19.249716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 78/148] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:19.257643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744f8fc-9bf8-4aeb-ae2c-899b738b7e21 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:19.399421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744f8fc-9bf8-4aeb-ae2c-899b738b7e21 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:19.443520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1e17e-05ac-4515-810e-ff62d5d13113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:19.444207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e17e-05ac-4515-810e-ff62d5d13113 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:19.444503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e17e-05ac-4515-810e-ff62d5d13113 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:19.444787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e17e-05ac-4515-810e-ff62d5d13113 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:19.445542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1e17e-05ac-4515-810e-ff62d5d13113 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:19.445845 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1e17e-05ac-4515-810e-ff62d5d13113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:19.447302 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1e17e-05ac-4515-810e-ff62d5d13113 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:19.460125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 71/149] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:19.465899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48b697-477c-44f5-8ec7-de8e91642ee3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:19.490831 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8acbb1-e1fa-4302-9a0a-3cc81504e1fd None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:19.491155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8acbb1-e1fa-4302-9a0a-3cc81504e1fd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:19.491388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8acbb1-e1fa-4302-9a0a-3cc81504e1fd None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:19.493861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 72/150] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:19.508410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737545d5-a461-46f1-8a6e-e393ba82efe5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:19.508725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737545d5-a461-46f1-8a6e-e393ba82efe5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:19.508997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737545d5-a461-46f1-8a6e-e393ba82efe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:19.533046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-737545d5-a461-46f1-8a6e-e393ba82efe5 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:19.534237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-737545d5-a461-46f1-8a6e-e393ba82efe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:19.650932 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48b697-477c-44f5-8ec7-de8e91642ee3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:19.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 73/151] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Apr 29 18:29:19.695735 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b7c17-ee8c-4b47-a61e-22826be3f7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:19.696692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b7c17-ee8c-4b47-a61e-22826be3f7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:19.696970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b7c17-ee8c-4b47-a61e-22826be3f7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:19.697280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b7c17-ee8c-4b47-a61e-22826be3f7f0 None admin] PATH_INFO: `/v3/users/c920286345e24cc09a50a6ef9210d7fc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:19.698486 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b7c17-ee8c-4b47-a61e-22826be3f7f0 None admin] RBAC: Authorizing `identity:get_user(user_id=c920286345e24cc09a50a6ef9210d7fc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:19.699953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b7c17-ee8c-4b47-a61e-22826be3f7f0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:19.701627 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 79/152] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:19 2022] GET /identity/v3/users/c920286345e24cc09a50a6ef9210d7fc => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:19.707361 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737545d5-a461-46f1-8a6e-e393ba82efe5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:19.720812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b7c17-ee8c-4b47-a61e-22826be3f7f0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:19.849111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737545d5-a461-46f1-8a6e-e393ba82efe5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:19.865226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b7c17-ee8c-4b47-a61e-22826be3f7f0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:19.888595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b39b5-acd4-4e0a-a9d7-8d95c28b57f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:19.889265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b39b5-acd4-4e0a-a9d7-8d95c28b57f0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:19.889544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b39b5-acd4-4e0a-a9d7-8d95c28b57f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:19.889815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b39b5-acd4-4e0a-a9d7-8d95c28b57f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:19.907458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3e14a-787f-46c0-bf32-122ba2905723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:19.908820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3e14a-787f-46c0-bf32-122ba2905723 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:19.909543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3e14a-787f-46c0-bf32-122ba2905723 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:19.910094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3e14a-787f-46c0-bf32-122ba2905723 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:19.923623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3e14a-787f-46c0-bf32-122ba2905723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-51a3e14a-787f-46c0-bf32-122ba2905723 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:19.932813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:19.933653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:19.934404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 80/153] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:19.935503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd34ea2-db1a-48da-a7fc-cb781982b933 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:19.935503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd34ea2-db1a-48da-a7fc-cb781982b933 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:19.935781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd34ea2-db1a-48da-a7fc-cb781982b933 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:19.937056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 81/154] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:19.941714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd34ea2-db1a-48da-a7fc-cb781982b933 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:19.965991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b39b5-acd4-4e0a-a9d7-8d95c28b57f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:19.967867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b39b5-acd4-4e0a-a9d7-8d95c28b57f0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:20.058062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd34ea2-db1a-48da-a7fc-cb781982b933 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:20.075341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 74/155] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Apr 29 18:29:20.081251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec7bc3-206d-451d-adb1-8c54875227b7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.081550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec7bc3-206d-451d-adb1-8c54875227b7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.081830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec7bc3-206d-451d-adb1-8c54875227b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.091244 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82fa85-332f-4187-b154-3d5ba7a9c5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:20.091920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82fa85-332f-4187-b154-3d5ba7a9c5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.092197 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82fa85-332f-4187-b154-3d5ba7a9c5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.092466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82fa85-332f-4187-b154-3d5ba7a9c5f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.093236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82fa85-332f-4187-b154-3d5ba7a9c5f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:20.093899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82fa85-332f-4187-b154-3d5ba7a9c5f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:20.095273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82fa85-332f-4187-b154-3d5ba7a9c5f4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:20.098440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7bec7bc3-206d-451d-adb1-8c54875227b7 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:20.099614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7bec7bc3-206d-451d-adb1-8c54875227b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:20.103718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 82/156] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:20.105071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82fa85-332f-4187-b154-3d5ba7a9c5f4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:20.166049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 75/157] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:20.167115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec7bc3-206d-451d-adb1-8c54875227b7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:20.209605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82fa85-332f-4187-b154-3d5ba7a9c5f4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:20.242020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e15eb-b895-4e2c-ac49-be01b9f1e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:20.242891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e15eb-b895-4e2c-ac49-be01b9f1e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.243165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e15eb-b895-4e2c-ac49-be01b9f1e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.243429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e15eb-b895-4e2c-ac49-be01b9f1e9c7 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.244688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e15eb-b895-4e2c-ac49-be01b9f1e9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:20.246271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e15eb-b895-4e2c-ac49-be01b9f1e9c7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:20.252158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 83/158] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:20 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:20.252158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57803d5-500a-4127-b79c-9dd8ea064199 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.252158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57803d5-500a-4127-b79c-9dd8ea064199 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.252158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57803d5-500a-4127-b79c-9dd8ea064199 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.252158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 84/159] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:20.252158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57803d5-500a-4127-b79c-9dd8ea064199 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:20.269040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec7bc3-206d-451d-adb1-8c54875227b7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:20.303444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a513f-7200-4e8d-819e-ef3227cf16ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:20.304605 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a513f-7200-4e8d-819e-ef3227cf16ed None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.304889 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a513f-7200-4e8d-819e-ef3227cf16ed None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.305406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a513f-7200-4e8d-819e-ef3227cf16ed None admin] PATH_INFO: `/v3/users/c920286345e24cc09a50a6ef9210d7fc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.307387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a513f-7200-4e8d-819e-ef3227cf16ed None admin] RBAC: Authorizing `identity:get_user(user_id=c920286345e24cc09a50a6ef9210d7fc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:20.309347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a513f-7200-4e8d-819e-ef3227cf16ed None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:20.316387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 76/160] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:20 2022] GET /identity/v3/users/c920286345e24cc09a50a6ef9210d7fc => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:20.316387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a513f-7200-4e8d-819e-ef3227cf16ed None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:20.348301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57803d5-500a-4127-b79c-9dd8ea064199 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:20.380995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfce74-c1f4-478c-8bcb-63f7f6633e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:20.381855 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfce74-c1f4-478c-8bcb-63f7f6633e68 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.382283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfce74-c1f4-478c-8bcb-63f7f6633e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.382408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfce74-c1f4-478c-8bcb-63f7f6633e68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.383667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfce74-c1f4-478c-8bcb-63f7f6633e68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:20.385584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfce74-c1f4-478c-8bcb-63f7f6633e68 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:20.387155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 85/161] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:20.388322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfb88f-aa7f-4811-a299-efecebb692ba None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.388604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfb88f-aa7f-4811-a299-efecebb692ba None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.388872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfb88f-aa7f-4811-a299-efecebb692ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.406377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-16dfb88f-aa7f-4811-a299-efecebb692ba None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:20.407599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-16dfb88f-aa7f-4811-a299-efecebb692ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:20.430531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a513f-7200-4e8d-819e-ef3227cf16ed None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:20.465917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768dc1d-61fb-4566-b62f-0e06ae9412b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:20.466942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768dc1d-61fb-4566-b62f-0e06ae9412b7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.467437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768dc1d-61fb-4566-b62f-0e06ae9412b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.467742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768dc1d-61fb-4566-b62f-0e06ae9412b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.469010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768dc1d-61fb-4566-b62f-0e06ae9412b7 None admin] RBAC: Adding query filter params (role.id=3385b41b990e4c6b88d77260a7822c7e, scope.project.id=7a48da3b90914e098e862c1aadf1620b, user.id=c920286345e24cc09a50a6ef9210d7fc) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:20.469320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768dc1d-61fb-4566-b62f-0e06ae9412b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:20.470763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768dc1d-61fb-4566-b62f-0e06ae9412b7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:20.474044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 86/162] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:20.474775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfb88f-aa7f-4811-a299-efecebb692ba None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:20.487661 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 77/163] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:20 2022] GET /identity/v3/role_assignments?user.id=c920286345e24cc09a50a6ef9210d7fc&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:20.497064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768dc1d-61fb-4566-b62f-0e06ae9412b7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:20.605447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfb88f-aa7f-4811-a299-efecebb692ba None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:20.617094 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768dc1d-61fb-4566-b62f-0e06ae9412b7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:20.641831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fa016-a6f9-47cf-8c5a-5e413506438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:20.642494 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fa016-a6f9-47cf-8c5a-5e413506438e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.642771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fa016-a6f9-47cf-8c5a-5e413506438e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.643041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fa016-a6f9-47cf-8c5a-5e413506438e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.651391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe55d5-58e3-4068-b4a6-2bd407b2474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:20.652078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe55d5-58e3-4068-b4a6-2bd407b2474e None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.652353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe55d5-58e3-4068-b4a6-2bd407b2474e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.652654 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe55d5-58e3-4068-b4a6-2bd407b2474e None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.653382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe55d5-58e3-4068-b4a6-2bd407b2474e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:20.654856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe55d5-58e3-4068-b4a6-2bd407b2474e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:20.676514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 78/164] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:29:20 2022] POST /identity/v3/users => generated 283 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:20.684522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe55d5-58e3-4068-b4a6-2bd407b2474e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:20.714999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fa016-a6f9-47cf-8c5a-5e413506438e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:20.716534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fa016-a6f9-47cf-8c5a-5e413506438e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:20.802029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe55d5-58e3-4068-b4a6-2bd407b2474e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:20.823680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 87/165] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:20.825130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f23c35-f870-449c-a187-1afcbdbb333a None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.825391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f23c35-f870-449c-a187-1afcbdbb333a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.825683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f23c35-f870-449c-a187-1afcbdbb333a None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.827211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 88/166] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:20.832107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f23c35-f870-449c-a187-1afcbdbb333a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:20.846498 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7322d-e68c-4f1e-9180-fb8185ea0264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:20.847320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7322d-e68c-4f1e-9180-fb8185ea0264 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.847597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7322d-e68c-4f1e-9180-fb8185ea0264 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.847854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7322d-e68c-4f1e-9180-fb8185ea0264 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.860352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7322d-e68c-4f1e-9180-fb8185ea0264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8df7322d-e68c-4f1e-9180-fb8185ea0264 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:20.868291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:20.869329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:20.870098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:20.875617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 79/167] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:20.876882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3581b-fa0d-4536-b0d0-383bbd31badd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.877173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3581b-fa0d-4536-b0d0-383bbd31badd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.877450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3581b-fa0d-4536-b0d0-383bbd31badd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.896179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3ed3581b-fa0d-4536-b0d0-383bbd31badd None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:20.897604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3ed3581b-fa0d-4536-b0d0-383bbd31badd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:20.939273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f23c35-f870-449c-a187-1afcbdbb333a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:20.976817 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 80/168] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:20.977690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3581b-fa0d-4536-b0d0-383bbd31badd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:20.981098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54706d-c705-4e3e-b2da-7016bacff6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:20.981981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54706d-c705-4e3e-b2da-7016bacff6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:20.982254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54706d-c705-4e3e-b2da-7016bacff6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:20.982523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54706d-c705-4e3e-b2da-7016bacff6a6 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:20.983700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54706d-c705-4e3e-b2da-7016bacff6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:20.985194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54706d-c705-4e3e-b2da-7016bacff6a6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:20.986892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 89/169] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:20 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:20.993583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54706d-c705-4e3e-b2da-7016bacff6a6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:21.098887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3581b-fa0d-4536-b0d0-383bbd31badd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:21.104790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54706d-c705-4e3e-b2da-7016bacff6a6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:21.144249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efed01d-dedb-47bc-8f48-3e4d2f218838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:21.145616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efed01d-dedb-47bc-8f48-3e4d2f218838 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:21.145901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efed01d-dedb-47bc-8f48-3e4d2f218838 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:21.146097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc6cd0-60f1-49ff-bb63-bedc92b0ecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:21.146281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efed01d-dedb-47bc-8f48-3e4d2f218838 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:21.146763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc6cd0-60f1-49ff-bb63-bedc92b0ecd5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:21.147043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc6cd0-60f1-49ff-bb63-bedc92b0ecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:21.147146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efed01d-dedb-47bc-8f48-3e4d2f218838 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:21.147313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc6cd0-60f1-49ff-bb63-bedc92b0ecd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:21.147403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efed01d-dedb-47bc-8f48-3e4d2f218838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:21.148135 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc6cd0-60f1-49ff-bb63-bedc92b0ecd5 None admin] RBAC: Adding query filter params (role.id=ec08a86158a44ee2b82951b44b228df2, scope.project.id=7a48da3b90914e098e862c1aadf1620b, user.id=c920286345e24cc09a50a6ef9210d7fc) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:21.148439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc6cd0-60f1-49ff-bb63-bedc92b0ecd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:21.148882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efed01d-dedb-47bc-8f48-3e4d2f218838 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:21.149925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc6cd0-60f1-49ff-bb63-bedc92b0ecd5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:21.158020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 90/170] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:20 2022] GET /identity/v3/role_assignments?user.id=c920286345e24cc09a50a6ef9210d7fc&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=ec08a86158a44ee2b82951b44b228df2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:21.158686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 81/171] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:21.158926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc6cd0-60f1-49ff-bb63-bedc92b0ecd5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:21.166120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efed01d-dedb-47bc-8f48-3e4d2f218838 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:21.296136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efed01d-dedb-47bc-8f48-3e4d2f218838 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:21.297306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc6cd0-60f1-49ff-bb63-bedc92b0ecd5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:21.335698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1a2e2-e29e-4d61-b1a7-673776624c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:21.336641 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a2e2-e29e-4d61-b1a7-673776624c26 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:21.336922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a2e2-e29e-4d61-b1a7-673776624c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:21.337194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1a2e2-e29e-4d61-b1a7-673776624c26 None admin] PATH_INFO: `/v3/users/e4c0b3cc764c4805a21f8d15a56d137f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:21.338194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acfb1a-e604-410a-abf6-ed510a2dd9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:21.338605 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1a2e2-e29e-4d61-b1a7-673776624c26 None admin] RBAC: Authorizing `identity:get_user(user_id=e4c0b3cc764c4805a21f8d15a56d137f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:21.339576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acfb1a-e604-410a-abf6-ed510a2dd9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:21.340079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acfb1a-e604-410a-abf6-ed510a2dd9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:21.340179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1a2e2-e29e-4d61-b1a7-673776624c26 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:21.340662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acfb1a-e604-410a-abf6-ed510a2dd9f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:21.341978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 82/172] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:21 2022] GET /identity/v3/users/e4c0b3cc764c4805a21f8d15a56d137f => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:21.349334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1a2e2-e29e-4d61-b1a7-673776624c26 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:21.362470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acfb1a-e604-410a-abf6-ed510a2dd9f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-01acfb1a-e604-410a-abf6-ed510a2dd9f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:21.371191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:21.372281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:21.373132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:21.375715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 91/173] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:21.382265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acfb1a-e604-410a-abf6-ed510a2dd9f1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:21.487443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1a2e2-e29e-4d61-b1a7-673776624c26 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:21.517983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acfb1a-e604-410a-abf6-ed510a2dd9f1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:21.527477 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fd1da-1e06-43c2-bac9-cd7259ec514a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:21.528760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fd1da-1e06-43c2-bac9-cd7259ec514a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:21.529036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fd1da-1e06-43c2-bac9-cd7259ec514a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:21.529304 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fd1da-1e06-43c2-bac9-cd7259ec514a None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:21.531328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fd1da-1e06-43c2-bac9-cd7259ec514a None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:21.532552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fd1da-1e06-43c2-bac9-cd7259ec514a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:21.534619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 83/174] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:21 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:21.540290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fd1da-1e06-43c2-bac9-cd7259ec514a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:21.557544 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ab694-a5fa-4acf-89af-ceea7ba1885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:21.558560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ab694-a5fa-4acf-89af-ceea7ba1885a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:21.559036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ab694-a5fa-4acf-89af-ceea7ba1885a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:21.559530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ab694-a5fa-4acf-89af-ceea7ba1885a None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:21.560474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ab694-a5fa-4acf-89af-ceea7ba1885a None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:21.561036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ab694-a5fa-4acf-89af-ceea7ba1885a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:21.562615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ab694-a5fa-4acf-89af-ceea7ba1885a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:21.571486 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 92/175] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:21.579379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ab694-a5fa-4acf-89af-ceea7ba1885a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:21.682094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ab694-a5fa-4acf-89af-ceea7ba1885a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:21.708787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fd1da-1e06-43c2-bac9-cd7259ec514a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:21.716716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa3add-24b2-4959-9166-bcdbf47f2a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:21.718247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa3add-24b2-4959-9166-bcdbf47f2a43 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:21.719101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa3add-24b2-4959-9166-bcdbf47f2a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:21.719565 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa3add-24b2-4959-9166-bcdbf47f2a43 None admin] PATH_INFO: `/v3/users/c920286345e24cc09a50a6ef9210d7fc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:21.721009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa3add-24b2-4959-9166-bcdbf47f2a43 None admin] RBAC: Authorizing `identity:get_user(user_id=c920286345e24cc09a50a6ef9210d7fc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:21.722677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa3add-24b2-4959-9166-bcdbf47f2a43 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:21.724619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 93/176] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:21 2022] GET /identity/v3/users/c920286345e24cc09a50a6ef9210d7fc => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:21.730237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa3add-24b2-4959-9166-bcdbf47f2a43 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:21.772585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf22e1-c922-479a-80af-9680e3678e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:21.774963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf22e1-c922-479a-80af-9680e3678e18 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:21.777139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf22e1-c922-479a-80af-9680e3678e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:21.777638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf22e1-c922-479a-80af-9680e3678e18 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b/users/e4c0b3cc764c4805a21f8d15a56d137f/roles/c6d29efc5be34cc097e61371c95cd446` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:21.781544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf22e1-c922-479a-80af-9680e3678e18 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a48da3b90914e098e862c1aadf1620b, user_id=e4c0b3cc764c4805a21f8d15a56d137f, role_id=c6d29efc5be34cc097e61371c95cd446)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:21.783433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf22e1-c922-479a-80af-9680e3678e18 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:21.815031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 84/177] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:29:21 2022] PUT /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b/users/e4c0b3cc764c4805a21f8d15a56d137f/roles/c6d29efc5be34cc097e61371c95cd446 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:29:21.873235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa3add-24b2-4959-9166-bcdbf47f2a43 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:21.908756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dbfb3-6d49-4e28-a077-5c85a78d22eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:21.909789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dbfb3-6d49-4e28-a077-5c85a78d22eb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:21.910287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dbfb3-6d49-4e28-a077-5c85a78d22eb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:21.910748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dbfb3-6d49-4e28-a077-5c85a78d22eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:21.981350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dbfb3-6d49-4e28-a077-5c85a78d22eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:21.983188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dbfb3-6d49-4e28-a077-5c85a78d22eb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:22.085196 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 94/178] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:22.094033 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf22e1-c922-479a-80af-9680e3678e18 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:22.204152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf22e1-c922-479a-80af-9680e3678e18 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:22.242893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028a549-53a3-4dac-9442-076e627914a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:22.243766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a549-53a3-4dac-9442-076e627914a2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:22.244081 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a549-53a3-4dac-9442-076e627914a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:22.244348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a549-53a3-4dac-9442-076e627914a2 None admin] PATH_INFO: `/v3/projects/3cca2b4862f24dbb8647acba4b570f23` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:22.246803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028a549-53a3-4dac-9442-076e627914a2 None admin] RBAC: Authorizing `identity:get_project(project_id=3cca2b4862f24dbb8647acba4b570f23)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:22.248255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028a549-53a3-4dac-9442-076e627914a2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:22.250355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 85/179] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:22 2022] GET /identity/v3/projects/3cca2b4862f24dbb8647acba4b570f23 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:22.256251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dbfb3-6d49-4e28-a077-5c85a78d22eb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:22.359802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dbfb3-6d49-4e28-a077-5c85a78d22eb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:22.396665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e7033-84da-4c72-a45d-d79a4741239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:22.397733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e7033-84da-4c72-a45d-d79a4741239d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:22.398210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e7033-84da-4c72-a45d-d79a4741239d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:22.398675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e7033-84da-4c72-a45d-d79a4741239d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:22.399697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e7033-84da-4c72-a45d-d79a4741239d None admin] RBAC: Adding query filter params (role.id=3385b41b990e4c6b88d77260a7822c7e, scope.project.id=3cca2b4862f24dbb8647acba4b570f23, user.id=c920286345e24cc09a50a6ef9210d7fc) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:22.400230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e7033-84da-4c72-a45d-d79a4741239d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:22.402060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e7033-84da-4c72-a45d-d79a4741239d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:22.408879 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 95/180] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:22 2022] GET /identity/v3/role_assignments?user.id=c920286345e24cc09a50a6ef9210d7fc&scope.project.id=3cca2b4862f24dbb8647acba4b570f23&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:23.340867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6d567-d89e-4efd-814c-321eda01eb8e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:23.341423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6d567-d89e-4efd-814c-321eda01eb8e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:23.341423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6d567-d89e-4efd-814c-321eda01eb8e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:23.343027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 86/181] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:23.348872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24cf59-44e8-421e-b2a0-9824e77803b5 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:23.349725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24cf59-44e8-421e-b2a0-9824e77803b5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:23.350186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24cf59-44e8-421e-b2a0-9824e77803b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:23.366933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-2f24cf59-44e8-421e-b2a0-9824e77803b5 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:23.368403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2f24cf59-44e8-421e-b2a0-9824e77803b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:23.437798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 96/182] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:23.460448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6d567-d89e-4efd-814c-321eda01eb8e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:23.565293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6d567-d89e-4efd-814c-321eda01eb8e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:23.604986 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a8f2d-dcf8-4bdc-8c38-8d6aee3e0eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:23.606115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a8f2d-dcf8-4bdc-8c38-8d6aee3e0eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:23.606582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a8f2d-dcf8-4bdc-8c38-8d6aee3e0eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:23.607033 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a8f2d-dcf8-4bdc-8c38-8d6aee3e0eb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:23.617491 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a8f2d-dcf8-4bdc-8c38-8d6aee3e0eb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3d0a8f2d-dcf8-4bdc-8c38-8d6aee3e0eb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:23.628283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:23.629204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:23.629995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 87/183] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:29:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:23.635735 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24cf59-44e8-421e-b2a0-9824e77803b5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:23.724895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acfd84-fcc9-4c35-9332-b5e77b1552a9 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:23.725247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acfd84-fcc9-4c35-9332-b5e77b1552a9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:23.725446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acfd84-fcc9-4c35-9332-b5e77b1552a9 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:23.726827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 88/184] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:29:23.732000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7f17b-43c1-47e1-b7dc-2b9e54eeff1c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:23.732281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7f17b-43c1-47e1-b7dc-2b9e54eeff1c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:23.732622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7f17b-43c1-47e1-b7dc-2b9e54eeff1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:23.734904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24cf59-44e8-421e-b2a0-9824e77803b5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:23.750728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b4b7f17b-43c1-47e1-b7dc-2b9e54eeff1c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:23.751988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b4b7f17b-43c1-47e1-b7dc-2b9e54eeff1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:23.773968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7f6ad-242d-43d6-aae4-0aa17d040715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:23.775071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7f6ad-242d-43d6-aae4-0aa17d040715 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:23.775811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7f6ad-242d-43d6-aae4-0aa17d040715 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:23.776296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7f6ad-242d-43d6-aae4-0aa17d040715 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:23.777281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7f6ad-242d-43d6-aae4-0aa17d040715 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:23.777804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7f6ad-242d-43d6-aae4-0aa17d040715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:23.779487 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7f6ad-242d-43d6-aae4-0aa17d040715 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:23.788692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 97/185] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:23.800523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7f6ad-242d-43d6-aae4-0aa17d040715 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:23.831006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 89/186] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:23.854852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7f17b-43c1-47e1-b7dc-2b9e54eeff1c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:23.897641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7f6ad-242d-43d6-aae4-0aa17d040715 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:23.932385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54ee49-7308-44ad-92d7-f546f33e05ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:23.933313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54ee49-7308-44ad-92d7-f546f33e05ff None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:23.933589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54ee49-7308-44ad-92d7-f546f33e05ff None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:23.933854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54ee49-7308-44ad-92d7-f546f33e05ff None admin] PATH_INFO: `/v3/users/3aae69207ed9492db04a8c0333864a63` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:23.948575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54ee49-7308-44ad-92d7-f546f33e05ff None admin] RBAC: Authorizing `identity:get_user(user_id=3aae69207ed9492db04a8c0333864a63)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:23.950041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54ee49-7308-44ad-92d7-f546f33e05ff None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:23.953667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7f17b-43c1-47e1-b7dc-2b9e54eeff1c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:23.956286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 98/187] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:23 2022] GET /identity/v3/users/3aae69207ed9492db04a8c0333864a63 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:23.957497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83357d-f90f-4f0f-bb40-14a1626ac097 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:23.957782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83357d-f90f-4f0f-bb40-14a1626ac097 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:23.958059 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83357d-f90f-4f0f-bb40-14a1626ac097 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:23.959713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 99/188] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:23.960249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83357d-f90f-4f0f-bb40-14a1626ac097 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:23.992324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53754461-5060-4f33-8a25-6318325b66bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:23.993165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53754461-5060-4f33-8a25-6318325b66bf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:23.993442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53754461-5060-4f33-8a25-6318325b66bf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:23.993710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53754461-5060-4f33-8a25-6318325b66bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.005953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53754461-5060-4f33-8a25-6318325b66bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-53754461-5060-4f33-8a25-6318325b66bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:24.017095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:24.017990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:24.018775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:24.019642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 90/189] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:24.020949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a2913-ed81-49ad-af7d-51a318c2641a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.021235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a2913-ed81-49ad-af7d-51a318c2641a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.021509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a2913-ed81-49ad-af7d-51a318c2641a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.038296 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-421a2913-ed81-49ad-af7d-51a318c2641a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:24.039463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-421a2913-ed81-49ad-af7d-51a318c2641a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:24.061596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83357d-f90f-4f0f-bb40-14a1626ac097 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:24.096190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc869e-1389-401f-bddb-2daa1e09dfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:24.097245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc869e-1389-401f-bddb-2daa1e09dfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.097721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc869e-1389-401f-bddb-2daa1e09dfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.098166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc869e-1389-401f-bddb-2daa1e09dfe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.103749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 91/190] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:24.104617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a2913-ed81-49ad-af7d-51a318c2641a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:24.166104 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc869e-1389-401f-bddb-2daa1e09dfe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:24.167473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc869e-1389-401f-bddb-2daa1e09dfe7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:24.211724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a2913-ed81-49ad-af7d-51a318c2641a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:24.245127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7acab-5e7f-4309-bb9a-5d51090eebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:24.245797 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7acab-5e7f-4309-bb9a-5d51090eebb9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.246110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7acab-5e7f-4309-bb9a-5d51090eebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.246332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7acab-5e7f-4309-bb9a-5d51090eebb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.247069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7acab-5e7f-4309-bb9a-5d51090eebb9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:24.247367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7acab-5e7f-4309-bb9a-5d51090eebb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:24.248776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7acab-5e7f-4309-bb9a-5d51090eebb9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:24.260917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 92/191] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:24.260917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7acab-5e7f-4309-bb9a-5d51090eebb9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:24.270098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 100/192] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:24.270880 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc869e-1389-401f-bddb-2daa1e09dfe7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:24.361953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7acab-5e7f-4309-bb9a-5d51090eebb9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:24.370131 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc869e-1389-401f-bddb-2daa1e09dfe7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:24.393730 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0ccdb-4ce9-4048-8590-fa84a7008e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:24.394520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0ccdb-4ce9-4048-8590-fa84a7008e75 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.394797 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0ccdb-4ce9-4048-8590-fa84a7008e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.395097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0ccdb-4ce9-4048-8590-fa84a7008e75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.396878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0ccdb-4ce9-4048-8590-fa84a7008e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:24.398289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0ccdb-4ce9-4048-8590-fa84a7008e75 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:24.400137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 93/193] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:24.400932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0ccdb-4ce9-4048-8590-fa84a7008e75 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:24.405963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae2715-103b-463c-9ccf-90de889d576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:24.406846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae2715-103b-463c-9ccf-90de889d576c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.407119 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae2715-103b-463c-9ccf-90de889d576c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.407386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae2715-103b-463c-9ccf-90de889d576c None admin] PATH_INFO: `/v3/users/3aae69207ed9492db04a8c0333864a63` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.408590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae2715-103b-463c-9ccf-90de889d576c None admin] RBAC: Authorizing `identity:get_user(user_id=3aae69207ed9492db04a8c0333864a63)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:24.410044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae2715-103b-463c-9ccf-90de889d576c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:24.411647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 101/194] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:24 2022] GET /identity/v3/users/3aae69207ed9492db04a8c0333864a63 => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:24.413321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae2715-103b-463c-9ccf-90de889d576c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:24.500156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0ccdb-4ce9-4048-8590-fa84a7008e75 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:24.505548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae2715-103b-463c-9ccf-90de889d576c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:24.531337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ba464-2e6b-47db-9206-f4972e80f6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:24.532200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ba464-2e6b-47db-9206-f4972e80f6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.532479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ba464-2e6b-47db-9206-f4972e80f6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.532781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ba464-2e6b-47db-9206-f4972e80f6b9 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.533955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ba464-2e6b-47db-9206-f4972e80f6b9 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:24.535417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ba464-2e6b-47db-9206-f4972e80f6b9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:24.536789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772932cc-7547-4bb2-ba9c-657cecd40af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:24.537147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 94/195] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:24 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:24.537975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ba464-2e6b-47db-9206-f4972e80f6b9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:24.542349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772932cc-7547-4bb2-ba9c-657cecd40af7 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.542645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772932cc-7547-4bb2-ba9c-657cecd40af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.542922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772932cc-7547-4bb2-ba9c-657cecd40af7 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.543644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772932cc-7547-4bb2-ba9c-657cecd40af7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:24.545116 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772932cc-7547-4bb2-ba9c-657cecd40af7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:24.565300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 102/196] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:29:24 2022] POST /identity/v3/users => generated 297 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:24.565300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cbe8-78cd-42a5-86c0-098fbbeaa821 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.565300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cbe8-78cd-42a5-86c0-098fbbeaa821 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.565300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cbe8-78cd-42a5-86c0-098fbbeaa821 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.565300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 103/197] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:24.565300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170cbe8-78cd-42a5-86c0-098fbbeaa821 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:24.645659 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ba464-2e6b-47db-9206-f4972e80f6b9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:24.680837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170cbe8-78cd-42a5-86c0-098fbbeaa821 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:24.682282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d04589-9a3f-4837-a0c2-5e9797a1898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:24.682954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d04589-9a3f-4837-a0c2-5e9797a1898d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.683229 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d04589-9a3f-4837-a0c2-5e9797a1898d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.683501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d04589-9a3f-4837-a0c2-5e9797a1898d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.715937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c7b9c-68e8-4a70-9fac-25cc708c729d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:24.716987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c7b9c-68e8-4a70-9fac-25cc708c729d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.717444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c7b9c-68e8-4a70-9fac-25cc708c729d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.717848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c7b9c-68e8-4a70-9fac-25cc708c729d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.719236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c7b9c-68e8-4a70-9fac-25cc708c729d None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=24cf0228dd084700bf349d5310285eb0, user.id=3aae69207ed9492db04a8c0333864a63) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:24.719681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c7b9c-68e8-4a70-9fac-25cc708c729d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:24.721307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c7b9c-68e8-4a70-9fac-25cc708c729d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:24.727676 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 104/198] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:24 2022] GET /identity/v3/role_assignments?user.id=3aae69207ed9492db04a8c0333864a63&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=e55ef11844a742f5889243f189bd59bc => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:24.734249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccd2b7-c55c-4c71-83d8-0bc3d1cfd07e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:24.734611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccd2b7-c55c-4c71-83d8-0bc3d1cfd07e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:24.734807 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccd2b7-c55c-4c71-83d8-0bc3d1cfd07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:24.757274 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8bccd2b7-c55c-4c71-83d8-0bc3d1cfd07e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:24.758447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8bccd2b7-c55c-4c71-83d8-0bc3d1cfd07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:24.761156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d04589-9a3f-4837-a0c2-5e9797a1898d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:24.762992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d04589-9a3f-4837-a0c2-5e9797a1898d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:24.848794 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 105/199] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:24.872356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccd2b7-c55c-4c71-83d8-0bc3d1cfd07e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:24.894410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 95/200] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:24.901011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d04589-9a3f-4837-a0c2-5e9797a1898d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:25.019485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccd2b7-c55c-4c71-83d8-0bc3d1cfd07e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:25.029141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d04589-9a3f-4837-a0c2-5e9797a1898d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:25.058913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fbaf6-fbc1-4ee0-8460-93472f3e4afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:25.059859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fbaf6-fbc1-4ee0-8460-93472f3e4afa None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:25.060200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fbaf6-fbc1-4ee0-8460-93472f3e4afa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:25.060556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fbaf6-fbc1-4ee0-8460-93472f3e4afa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:25.073018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5fbaf6-fbc1-4ee0-8460-93472f3e4afa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:25.073316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cc3eb-2497-42e9-a399-8e0dfc9ee243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:25.074385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cc3eb-2497-42e9-a399-8e0dfc9ee243 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:25.074830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cc3eb-2497-42e9-a399-8e0dfc9ee243 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:25.075284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cc3eb-2497-42e9-a399-8e0dfc9ee243 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:25.076743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cc3eb-2497-42e9-a399-8e0dfc9ee243 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:25.078379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cc3eb-2497-42e9-a399-8e0dfc9ee243 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:25.080265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 96/201] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:24 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ff5fbaf6-fbc1-4ee0-8460-93472f3e4afa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:25.082883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:25.083771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:25.084881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 106/202] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:25.090200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fbaf6-fbc1-4ee0-8460-93472f3e4afa None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:25.090335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cc3eb-2497-42e9-a399-8e0dfc9ee243 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:25.199193 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cc3eb-2497-42e9-a399-8e0dfc9ee243 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:25.235710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c601ea-92bf-4744-b5c1-7076ef43b8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:25.236714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c601ea-92bf-4744-b5c1-7076ef43b8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:25.237161 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c601ea-92bf-4744-b5c1-7076ef43b8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:25.237598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c601ea-92bf-4744-b5c1-7076ef43b8f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:25.238590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c601ea-92bf-4744-b5c1-7076ef43b8f4 None admin] RBAC: Adding query filter params (role.id=ec08a86158a44ee2b82951b44b228df2, scope.project.id=24cf0228dd084700bf349d5310285eb0, user.id=3aae69207ed9492db04a8c0333864a63) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:25.239090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c601ea-92bf-4744-b5c1-7076ef43b8f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:25.240712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c601ea-92bf-4744-b5c1-7076ef43b8f4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:25.247116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 97/203] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:25 2022] GET /identity/v3/role_assignments?user.id=3aae69207ed9492db04a8c0333864a63&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=ec08a86158a44ee2b82951b44b228df2 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:25.291300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fbaf6-fbc1-4ee0-8460-93472f3e4afa None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:25.326219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9efbf-e88a-4fd7-995c-a3e5c34265b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:25.326965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9efbf-e88a-4fd7-995c-a3e5c34265b1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:25.327289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9efbf-e88a-4fd7-995c-a3e5c34265b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:25.327569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9efbf-e88a-4fd7-995c-a3e5c34265b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:25.328384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9efbf-e88a-4fd7-995c-a3e5c34265b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:25.328835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9efbf-e88a-4fd7-995c-a3e5c34265b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:25.330289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9efbf-e88a-4fd7-995c-a3e5c34265b1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:25.341448 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 107/204] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:25.347600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c601ea-92bf-4744-b5c1-7076ef43b8f4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:25.471467 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c601ea-92bf-4744-b5c1-7076ef43b8f4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:25.508212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c5157-8d06-485c-bc6c-16eb36026813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:25.509185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c5157-8d06-485c-bc6c-16eb36026813 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:25.509468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c5157-8d06-485c-bc6c-16eb36026813 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:25.509779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c5157-8d06-485c-bc6c-16eb36026813 None admin] PATH_INFO: `/v3/users/e4c0b3cc764c4805a21f8d15a56d137f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:25.510995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c5157-8d06-485c-bc6c-16eb36026813 None admin] RBAC: Authorizing `identity:get_user(user_id=e4c0b3cc764c4805a21f8d15a56d137f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:25.512456 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c5157-8d06-485c-bc6c-16eb36026813 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:25.514151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 98/205] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:25 2022] GET /identity/v3/users/e4c0b3cc764c4805a21f8d15a56d137f => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:25.520632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9efbf-e88a-4fd7-995c-a3e5c34265b1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:25.623336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9efbf-e88a-4fd7-995c-a3e5c34265b1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:25.656852 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e4310-1cb4-4e5a-803c-ef3a06f93556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:25.657519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e4310-1cb4-4e5a-803c-ef3a06f93556 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:25.657852 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e4310-1cb4-4e5a-803c-ef3a06f93556 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:25.658125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e4310-1cb4-4e5a-803c-ef3a06f93556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:25.727681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e4310-1cb4-4e5a-803c-ef3a06f93556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:25.729289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e4310-1cb4-4e5a-803c-ef3a06f93556 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:25.843134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 108/206] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:25.849266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c5157-8d06-485c-bc6c-16eb36026813 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:25.949525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c5157-8d06-485c-bc6c-16eb36026813 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:25.986100 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d107342-c307-429b-858a-a7345de76e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:25.986984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d107342-c307-429b-858a-a7345de76e18 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:25.987265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d107342-c307-429b-858a-a7345de76e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:25.987537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d107342-c307-429b-858a-a7345de76e18 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:25.988765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d107342-c307-429b-858a-a7345de76e18 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:25.990225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d107342-c307-429b-858a-a7345de76e18 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:25.992246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 99/207] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:25 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:25.998201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e4310-1cb4-4e5a-803c-ef3a06f93556 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:26.127186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e4310-1cb4-4e5a-803c-ef3a06f93556 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:26.170412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddeface-89c7-4158-9186-ef1b0b8d0c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:26.171083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddeface-89c7-4158-9186-ef1b0b8d0c71 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:26.171365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddeface-89c7-4158-9186-ef1b0b8d0c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:26.171643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddeface-89c7-4158-9186-ef1b0b8d0c71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:26.172467 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddeface-89c7-4158-9186-ef1b0b8d0c71 None admin] RBAC: Adding query filter params (role.id=c6d29efc5be34cc097e61371c95cd446, scope.project.id=7a48da3b90914e098e862c1aadf1620b, user.id=e4c0b3cc764c4805a21f8d15a56d137f) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:26.172799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddeface-89c7-4158-9186-ef1b0b8d0c71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:26.174237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddeface-89c7-4158-9186-ef1b0b8d0c71 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:26.185642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 109/208] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:25 2022] GET /identity/v3/role_assignments?user.id=e4c0b3cc764c4805a21f8d15a56d137f&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=c6d29efc5be34cc097e61371c95cd446 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:27.170101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4050e6-362b-4cab-8c32-ab1eca3ef146 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.171116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4050e6-362b-4cab-8c32-ab1eca3ef146 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.171594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4050e6-362b-4cab-8c32-ab1eca3ef146 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.173219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 100/209] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:27.178839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec34cc7-8645-493f-a38e-604f289d9103 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.179679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec34cc7-8645-493f-a38e-604f289d9103 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.180136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec34cc7-8645-493f-a38e-604f289d9103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.197334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7ec34cc7-8645-493f-a38e-604f289d9103 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:27.198860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7ec34cc7-8645-493f-a38e-604f289d9103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:27.239990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc54287-659e-4735-8575-a5651923e8e1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.240645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc54287-659e-4735-8575-a5651923e8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.240971 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc54287-659e-4735-8575-a5651923e8e1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.242766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 101/210] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:29:27.248366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843230a3-090f-4cdc-91dc-1df8dd0bec0a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.248955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843230a3-090f-4cdc-91dc-1df8dd0bec0a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.249419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843230a3-090f-4cdc-91dc-1df8dd0bec0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.265647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 110/211] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:27.268864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-843230a3-090f-4cdc-91dc-1df8dd0bec0a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:27.270246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-843230a3-090f-4cdc-91dc-1df8dd0bec0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:27.288000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec34cc7-8645-493f-a38e-604f289d9103 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:27.356564 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 102/212] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:27.374174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843230a3-090f-4cdc-91dc-1df8dd0bec0a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:27.385890 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec34cc7-8645-493f-a38e-604f289d9103 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:27.423458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954c24d-898e-4451-896b-65d9397458e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:27.424181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c24d-898e-4451-896b-65d9397458e4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.424563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c24d-898e-4451-896b-65d9397458e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.424768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c24d-898e-4451-896b-65d9397458e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.435219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954c24d-898e-4451-896b-65d9397458e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-b954c24d-898e-4451-896b-65d9397458e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:27.442651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:27.443726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:27.444902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 111/213] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:29:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:27.450220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954c24d-898e-4451-896b-65d9397458e4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:27.480355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843230a3-090f-4cdc-91dc-1df8dd0bec0a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:27.517287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886eeca-94c7-4d38-bffc-4bc14854056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:27.518524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886eeca-94c7-4d38-bffc-4bc14854056d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.518998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886eeca-94c7-4d38-bffc-4bc14854056d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.519476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886eeca-94c7-4d38-bffc-4bc14854056d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.530304 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886eeca-94c7-4d38-bffc-4bc14854056d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e886eeca-94c7-4d38-bffc-4bc14854056d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:27.538293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:27.539164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:27.539973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:27.547356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 103/214] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:27.548907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886eeca-94c7-4d38-bffc-4bc14854056d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:27.561042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954c24d-898e-4451-896b-65d9397458e4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:27.594629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744ee8e-e0ea-4874-90af-15ca7eeb2135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:27.595328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ee8e-e0ea-4874-90af-15ca7eeb2135 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.595606 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ee8e-e0ea-4874-90af-15ca7eeb2135 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.595876 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ee8e-e0ea-4874-90af-15ca7eeb2135 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.596692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744ee8e-e0ea-4874-90af-15ca7eeb2135 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:27.596992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744ee8e-e0ea-4874-90af-15ca7eeb2135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:27.598436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744ee8e-e0ea-4874-90af-15ca7eeb2135 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:27.607979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 112/215] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:27.613456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744ee8e-e0ea-4874-90af-15ca7eeb2135 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:27.677238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886eeca-94c7-4d38-bffc-4bc14854056d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:27.712602 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ed078-3856-47ff-a5aa-509f622078a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:27.713503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed078-3856-47ff-a5aa-509f622078a6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.713761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed078-3856-47ff-a5aa-509f622078a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.714006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed078-3856-47ff-a5aa-509f622078a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.714751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ed078-3856-47ff-a5aa-509f622078a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:27.715053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ed078-3856-47ff-a5aa-509f622078a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:27.716455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ed078-3856-47ff-a5aa-509f622078a6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:27.724874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 104/216] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:27.726153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744ee8e-e0ea-4874-90af-15ca7eeb2135 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:27.726260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3be975-4903-477e-9908-32e15ce20fa3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.726458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3be975-4903-477e-9908-32e15ce20fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.726734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3be975-4903-477e-9908-32e15ce20fa3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.727957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 105/217] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:27.730919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3be975-4903-477e-9908-32e15ce20fa3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:27.763039 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e1639-0e57-4c62-a8eb-5a6e2a5af46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:27.763921 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e1639-0e57-4c62-a8eb-5a6e2a5af46a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.764198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e1639-0e57-4c62-a8eb-5a6e2a5af46a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.764467 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e1639-0e57-4c62-a8eb-5a6e2a5af46a None admin] PATH_INFO: `/v3/users/3aae69207ed9492db04a8c0333864a63` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.765788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e1639-0e57-4c62-a8eb-5a6e2a5af46a None admin] RBAC: Authorizing `identity:get_user(user_id=3aae69207ed9492db04a8c0333864a63)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:27.767230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e1639-0e57-4c62-a8eb-5a6e2a5af46a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:27.768914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 113/218] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:27 2022] GET /identity/v3/users/3aae69207ed9492db04a8c0333864a63 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:27.770110 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296255-d2f9-4a53-a94c-17853e084d15 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.770387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296255-d2f9-4a53-a94c-17853e084d15 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.770654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296255-d2f9-4a53-a94c-17853e084d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.786635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dc296255-d2f9-4a53-a94c-17853e084d15 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:27.787832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dc296255-d2f9-4a53-a94c-17853e084d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:27.834045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3be975-4903-477e-9908-32e15ce20fa3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:27.849960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 114/219] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:27.856677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc296255-d2f9-4a53-a94c-17853e084d15 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:27.889853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ffff4-b1ee-43b0-8b5f-6d0fb7e18164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:27.890734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ffff4-b1ee-43b0-8b5f-6d0fb7e18164 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.891013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ffff4-b1ee-43b0-8b5f-6d0fb7e18164 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.891282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ffff4-b1ee-43b0-8b5f-6d0fb7e18164 None admin] PATH_INFO: `/v3/users/9c033a34517942fd918cb10e1f210dc2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.906515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ffff4-b1ee-43b0-8b5f-6d0fb7e18164 None admin] RBAC: Authorizing `identity:get_user(user_id=9c033a34517942fd918cb10e1f210dc2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:27.908007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ffff4-b1ee-43b0-8b5f-6d0fb7e18164 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:27.909845 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 106/220] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:27 2022] GET /identity/v3/users/9c033a34517942fd918cb10e1f210dc2 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:27.913518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ffff4-b1ee-43b0-8b5f-6d0fb7e18164 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:27.958503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc296255-d2f9-4a53-a94c-17853e084d15 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:27.991194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00c5b8-6889-4ddc-ae6e-f113c904d67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:27.992065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00c5b8-6889-4ddc-ae6e-f113c904d67c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:27.992337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00c5b8-6889-4ddc-ae6e-f113c904d67c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:27.992645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00c5b8-6889-4ddc-ae6e-f113c904d67c None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:27.993805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00c5b8-6889-4ddc-ae6e-f113c904d67c None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:27.995386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00c5b8-6889-4ddc-ae6e-f113c904d67c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:27.997283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 115/221] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:27 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:27.997946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00c5b8-6889-4ddc-ae6e-f113c904d67c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:28.012203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ffff4-b1ee-43b0-8b5f-6d0fb7e18164 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:28.045119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e6c50-a827-475e-8c80-3c89bf06bc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:28.045927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e6c50-a827-475e-8c80-3c89bf06bc5a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:28.046208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e6c50-a827-475e-8c80-3c89bf06bc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:28.046481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e6c50-a827-475e-8c80-3c89bf06bc5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:28.047645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e6c50-a827-475e-8c80-3c89bf06bc5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:28.049131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e6c50-a827-475e-8c80-3c89bf06bc5a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:28.055209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 107/222] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:28.055209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e6c50-a827-475e-8c80-3c89bf06bc5a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:28.097231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00c5b8-6889-4ddc-ae6e-f113c904d67c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:28.128968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b6e4e-9e65-4ae1-b5fe-c5f6e477950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:28.129625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b6e4e-9e65-4ae1-b5fe-c5f6e477950f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:28.129903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b6e4e-9e65-4ae1-b5fe-c5f6e477950f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:28.130171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b6e4e-9e65-4ae1-b5fe-c5f6e477950f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:28.146655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e6c50-a827-475e-8c80-3c89bf06bc5a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:28.178874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2bdc-8e85-428c-b4b1-a7eb4d049587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:28.179752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2bdc-8e85-428c-b4b1-a7eb4d049587 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:28.180028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2bdc-8e85-428c-b4b1-a7eb4d049587 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:28.180277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2bdc-8e85-428c-b4b1-a7eb4d049587 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0/users/3aae69207ed9492db04a8c0333864a63/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:28.182550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2bdc-8e85-428c-b4b1-a7eb4d049587 None admin] RBAC: Authorizing `identity:create_grant(project_id=24cf0228dd084700bf349d5310285eb0, user_id=3aae69207ed9492db04a8c0333864a63, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:28.184000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2bdc-8e85-428c-b4b1-a7eb4d049587 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:28.195155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b6e4e-9e65-4ae1-b5fe-c5f6e477950f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:28.196654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b6e4e-9e65-4ae1-b5fe-c5f6e477950f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:28.197286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 108/223] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:29:28 2022] PUT /identity/v3/projects/24cf0228dd084700bf349d5310285eb0/users/3aae69207ed9492db04a8c0333864a63/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:28.198210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2bdc-8e85-428c-b4b1-a7eb4d049587 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:28.300902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2bdc-8e85-428c-b4b1-a7eb4d049587 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:28.307366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 116/224] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:28.308650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26a0ed-e892-47e9-9b49-0213166c4549 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:28.308938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26a0ed-e892-47e9-9b49-0213166c4549 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:28.309209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26a0ed-e892-47e9-9b49-0213166c4549 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:28.310649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 117/225] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:28.315488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26a0ed-e892-47e9-9b49-0213166c4549 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:28.345842 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584a0ce-16d6-42ee-92c8-02dc05faefae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:28.346514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584a0ce-16d6-42ee-92c8-02dc05faefae None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:28.346796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584a0ce-16d6-42ee-92c8-02dc05faefae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:28.347065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584a0ce-16d6-42ee-92c8-02dc05faefae None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:28.347807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584a0ce-16d6-42ee-92c8-02dc05faefae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:28.349386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584a0ce-16d6-42ee-92c8-02dc05faefae None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:28.371252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 109/226] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:29:28 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:28.378791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127370-ff1f-466a-883d-e77cde4bd0dc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:28.379790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127370-ff1f-466a-883d-e77cde4bd0dc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:28.380635 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34127370-ff1f-466a-883d-e77cde4bd0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:28.400158 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-34127370-ff1f-466a-883d-e77cde4bd0dc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:28.401890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34127370-ff1f-466a-883d-e77cde4bd0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:28.430317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26a0ed-e892-47e9-9b49-0213166c4549 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:28.467640 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e733067-f62a-4ec5-8e0c-097f4bf6ca77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:28.469289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e733067-f62a-4ec5-8e0c-097f4bf6ca77 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:28.469737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e733067-f62a-4ec5-8e0c-097f4bf6ca77 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:28.470204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e733067-f62a-4ec5-8e0c-097f4bf6ca77 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:28.471651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e733067-f62a-4ec5-8e0c-097f4bf6ca77 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:28.473351 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e733067-f62a-4ec5-8e0c-097f4bf6ca77 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:28.479363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 110/228] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:28.479680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 118/228] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:28 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:28.482290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34127370-ff1f-466a-883d-e77cde4bd0dc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:28.505806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e733067-f62a-4ec5-8e0c-097f4bf6ca77 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:28.589763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34127370-ff1f-466a-883d-e77cde4bd0dc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:28.619893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e733067-f62a-4ec5-8e0c-097f4bf6ca77 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:28.637462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d2c6-c82a-4ecc-bb65-c4e3ca4b62c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:28.638147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d2c6-c82a-4ecc-bb65-c4e3ca4b62c5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:28.638590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d2c6-c82a-4ecc-bb65-c4e3ca4b62c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:28.638872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d2c6-c82a-4ecc-bb65-c4e3ca4b62c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:28.640113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25d2c6-c82a-4ecc-bb65-c4e3ca4b62c5 None admin] RBAC: Adding query filter params (role.id=3385b41b990e4c6b88d77260a7822c7e, scope.project.id=24cf0228dd084700bf349d5310285eb0, user.id=9c033a34517942fd918cb10e1f210dc2) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:28.640712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25d2c6-c82a-4ecc-bb65-c4e3ca4b62c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:28.642338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25d2c6-c82a-4ecc-bb65-c4e3ca4b62c5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:28.649016 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 111/229] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:28 2022] GET /identity/v3/role_assignments?user.id=9c033a34517942fd918cb10e1f210dc2&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:28.661155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa242b-bbe9-4e85-bcde-79d8e2a6abe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:28.662000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa242b-bbe9-4e85-bcde-79d8e2a6abe3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:28.662277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa242b-bbe9-4e85-bcde-79d8e2a6abe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:28.663086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa242b-bbe9-4e85-bcde-79d8e2a6abe3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:28.677902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa242b-bbe9-4e85-bcde-79d8e2a6abe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-a5aa242b-bbe9-4e85-bcde-79d8e2a6abe3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:28.684954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:28.685909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:28.686694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 119/230] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:28.692419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d2c6-c82a-4ecc-bb65-c4e3ca4b62c5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:28.805255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d2c6-c82a-4ecc-bb65-c4e3ca4b62c5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:28.841961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7a4fa-e0d9-4cf3-9f33-34032217f72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:28.842568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7a4fa-e0d9-4cf3-9f33-34032217f72e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:28.842846 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7a4fa-e0d9-4cf3-9f33-34032217f72e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:28.843119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7a4fa-e0d9-4cf3-9f33-34032217f72e None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:28.844245 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7a4fa-e0d9-4cf3-9f33-34032217f72e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:28.844579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7a4fa-e0d9-4cf3-9f33-34032217f72e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:28.846016 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7a4fa-e0d9-4cf3-9f33-34032217f72e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:28.856390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 112/231] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:28.862316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa242b-bbe9-4e85-bcde-79d8e2a6abe3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:28.967856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa242b-bbe9-4e85-bcde-79d8e2a6abe3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:29.008008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57284b-112a-422a-ab5a-9e3f54043286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:29.008935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57284b-112a-422a-ab5a-9e3f54043286 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:29.009215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57284b-112a-422a-ab5a-9e3f54043286 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:29.009923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57284b-112a-422a-ab5a-9e3f54043286 None admin] PATH_INFO: `/v3/users/3aae69207ed9492db04a8c0333864a63` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:29.011184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57284b-112a-422a-ab5a-9e3f54043286 None admin] RBAC: Authorizing `identity:get_user(user_id=3aae69207ed9492db04a8c0333864a63)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:29.012682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57284b-112a-422a-ab5a-9e3f54043286 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:29.014328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 120/232] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:28 2022] GET /identity/v3/users/3aae69207ed9492db04a8c0333864a63 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:29.019906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7a4fa-e0d9-4cf3-9f33-34032217f72e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:29.139909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7a4fa-e0d9-4cf3-9f33-34032217f72e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:29.188674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761ef9b-98f9-40d9-a649-53c60f8bc4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:29.189542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761ef9b-98f9-40d9-a649-53c60f8bc4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:29.189822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761ef9b-98f9-40d9-a649-53c60f8bc4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:29.190091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761ef9b-98f9-40d9-a649-53c60f8bc4a6 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:29.197270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761ef9b-98f9-40d9-a649-53c60f8bc4a6 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:29.198732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761ef9b-98f9-40d9-a649-53c60f8bc4a6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:29.200800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 113/233] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:29 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:29.206809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57284b-112a-422a-ab5a-9e3f54043286 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:29.316552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57284b-112a-422a-ab5a-9e3f54043286 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:29.356049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b3afe-74ca-4d5f-9213-8f5aa88cca87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:29.356991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b3afe-74ca-4d5f-9213-8f5aa88cca87 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:29.357275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b3afe-74ca-4d5f-9213-8f5aa88cca87 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:29.357551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b3afe-74ca-4d5f-9213-8f5aa88cca87 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0/users/3aae69207ed9492db04a8c0333864a63/roles/ec08a86158a44ee2b82951b44b228df2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:29.359947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b3afe-74ca-4d5f-9213-8f5aa88cca87 None admin] RBAC: Authorizing `identity:create_grant(project_id=24cf0228dd084700bf349d5310285eb0, user_id=3aae69207ed9492db04a8c0333864a63, role_id=ec08a86158a44ee2b82951b44b228df2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:29.361442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b3afe-74ca-4d5f-9213-8f5aa88cca87 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:29.392263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 121/234] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:29:29 2022] PUT /identity/v3/projects/24cf0228dd084700bf349d5310285eb0/users/3aae69207ed9492db04a8c0333864a63/roles/ec08a86158a44ee2b82951b44b228df2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:30.839940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c51674-4110-48f9-9109-8319d00893e3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:30.840663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c51674-4110-48f9-9109-8319d00893e3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:30.840663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c51674-4110-48f9-9109-8319d00893e3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:30.842340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 114/235] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:30.848381 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f395fd-9f97-48b0-a194-1c71ca07f249 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:30.849060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f395fd-9f97-48b0-a194-1c71ca07f249 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:30.849060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f395fd-9f97-48b0-a194-1c71ca07f249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:30.871863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b4f395fd-9f97-48b0-a194-1c71ca07f249 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:30.873164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b4f395fd-9f97-48b0-a194-1c71ca07f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:30.963611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 122/236] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:31.010049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c51674-4110-48f9-9109-8319d00893e3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:31.043931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daba86-4849-46f7-930d-c4eb01b31fa7 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.044606 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daba86-4849-46f7-930d-c4eb01b31fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.045066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daba86-4849-46f7-930d-c4eb01b31fa7 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.046926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 123/237] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:29:31.051687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94166fa1-2ac1-4940-855f-9b655965393b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.052201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94166fa1-2ac1-4940-855f-9b655965393b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.052687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94166fa1-2ac1-4940-855f-9b655965393b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.083294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-94166fa1-2ac1-4940-855f-9b655965393b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:31.085987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-94166fa1-2ac1-4940-855f-9b655965393b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:31.186201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c51674-4110-48f9-9109-8319d00893e3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:31.200534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 124/238] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:31.244209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7316a5-dab4-443a-bf8b-5d9b7b03ae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:31.245108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7316a5-dab4-443a-bf8b-5d9b7b03ae47 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.245386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7316a5-dab4-443a-bf8b-5d9b7b03ae47 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.245657 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7316a5-dab4-443a-bf8b-5d9b7b03ae47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.261297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7316a5-dab4-443a-bf8b-5d9b7b03ae47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7e7316a5-dab4-443a-bf8b-5d9b7b03ae47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:31.268838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:31.269700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:31.270465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 115/239] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:29:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:31.273001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94166fa1-2ac1-4940-855f-9b655965393b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:31.276285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7316a5-dab4-443a-bf8b-5d9b7b03ae47 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:31.398222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7316a5-dab4-443a-bf8b-5d9b7b03ae47 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:31.440060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12259e0a-30db-4013-a713-c64843c57fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:31.441129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259e0a-30db-4013-a713-c64843c57fcc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.441417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259e0a-30db-4013-a713-c64843c57fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.441953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259e0a-30db-4013-a713-c64843c57fcc None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.442870 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12259e0a-30db-4013-a713-c64843c57fcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:31.443182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12259e0a-30db-4013-a713-c64843c57fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:31.445135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12259e0a-30db-4013-a713-c64843c57fcc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:31.446087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94166fa1-2ac1-4940-855f-9b655965393b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:31.455350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 116/240] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:31.457209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c4e5-23f7-44b9-b59d-19b0b7cf1b23 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.457877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c4e5-23f7-44b9-b59d-19b0b7cf1b23 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.458157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c4e5-23f7-44b9-b59d-19b0b7cf1b23 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.459961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 117/241] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:31.460812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37c4e5-23f7-44b9-b59d-19b0b7cf1b23 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:31.490077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459d2c4-a479-409d-9027-e04d08762395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:31.490966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d2c4-a479-409d-9027-e04d08762395 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.491247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d2c4-a479-409d-9027-e04d08762395 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.491516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459d2c4-a479-409d-9027-e04d08762395 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.503203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459d2c4-a479-409d-9027-e04d08762395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6459d2c4-a479-409d-9027-e04d08762395 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:31.511655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:31.512598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:31.513476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 125/242] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:31.514605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2a535-3bf2-4634-a0dc-c0959aca8b01 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.514605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2a535-3bf2-4634-a0dc-c0959aca8b01 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.514874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2a535-3bf2-4634-a0dc-c0959aca8b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.536306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-91d2a535-3bf2-4634-a0dc-c0959aca8b01 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:31.537566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-91d2a535-3bf2-4634-a0dc-c0959aca8b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:31.581940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37c4e5-23f7-44b9-b59d-19b0b7cf1b23 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:31.622391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 126/243] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:31.628636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2a535-3bf2-4634-a0dc-c0959aca8b01 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:31.634996 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5985ce-7d58-4e7b-b74b-658bfca7ba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:31.636183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5985ce-7d58-4e7b-b74b-658bfca7ba92 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.636695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5985ce-7d58-4e7b-b74b-658bfca7ba92 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.637189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5985ce-7d58-4e7b-b74b-658bfca7ba92 None admin] PATH_INFO: `/v3/users/3aae69207ed9492db04a8c0333864a63` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.641190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5985ce-7d58-4e7b-b74b-658bfca7ba92 None admin] RBAC: Authorizing `identity:get_user(user_id=3aae69207ed9492db04a8c0333864a63)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:31.642918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5985ce-7d58-4e7b-b74b-658bfca7ba92 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:31.645098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 118/244] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:31 2022] GET /identity/v3/users/3aae69207ed9492db04a8c0333864a63 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:31.650347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5985ce-7d58-4e7b-b74b-658bfca7ba92 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:31.721673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2a535-3bf2-4634-a0dc-c0959aca8b01 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:31.753399 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5985ce-7d58-4e7b-b74b-658bfca7ba92 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:31.754755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126e48d-048b-4e2d-84fb-0ca6b68cb5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:31.755432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126e48d-048b-4e2d-84fb-0ca6b68cb5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.755738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126e48d-048b-4e2d-84fb-0ca6b68cb5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.755977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126e48d-048b-4e2d-84fb-0ca6b68cb5b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.756742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126e48d-048b-4e2d-84fb-0ca6b68cb5b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:31.757042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126e48d-048b-4e2d-84fb-0ca6b68cb5b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:31.758432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126e48d-048b-4e2d-84fb-0ca6b68cb5b7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:31.766585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 127/245] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:31.767817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64dee4f-6d8c-40ab-b17b-8b045570fd07 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.768099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64dee4f-6d8c-40ab-b17b-8b045570fd07 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.768367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64dee4f-6d8c-40ab-b17b-8b045570fd07 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.769748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 128/246] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:31.770255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64dee4f-6d8c-40ab-b17b-8b045570fd07 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:31.793936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d765935-e976-431a-8afc-cbf94246188f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:31.794586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d765935-e976-431a-8afc-cbf94246188f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.794863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d765935-e976-431a-8afc-cbf94246188f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.795178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d765935-e976-431a-8afc-cbf94246188f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.860888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d765935-e976-431a-8afc-cbf94246188f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:31.862432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d765935-e976-431a-8afc-cbf94246188f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:31.872389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64dee4f-6d8c-40ab-b17b-8b045570fd07 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:31.906424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332810d2-a340-4224-b142-fd64fef148b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:31.907234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332810d2-a340-4224-b142-fd64fef148b1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.907516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332810d2-a340-4224-b142-fd64fef148b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.907787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332810d2-a340-4224-b142-fd64fef148b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.908975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332810d2-a340-4224-b142-fd64fef148b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:31.910428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332810d2-a340-4224-b142-fd64fef148b1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:31.912033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 129/247] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:31.912802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332810d2-a340-4224-b142-fd64fef148b1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:31.970882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 119/248] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Apr 29 18:29:31.972683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f86eeb-07c5-43c3-836d-45cc0b3345c7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:31.972909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f86eeb-07c5-43c3-836d-45cc0b3345c7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:31.973183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f86eeb-07c5-43c3-836d-45cc0b3345c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:31.990894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e9f86eeb-07c5-43c3-836d-45cc0b3345c7 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:31.992103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e9f86eeb-07c5-43c3-836d-45cc0b3345c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:32.012575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332810d2-a340-4224-b142-fd64fef148b1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:32.046151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95781f-c25e-4b7c-bb2a-93a561e46a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:32.047287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95781f-c25e-4b7c-bb2a-93a561e46a67 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.047721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95781f-c25e-4b7c-bb2a-93a561e46a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.047990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95781f-c25e-4b7c-bb2a-93a561e46a67 None admin] PATH_INFO: `/v3/users/22aa11f7dfd046698081a55635933992` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.057531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 120/249] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:32.061505 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f86eeb-07c5-43c3-836d-45cc0b3345c7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:32.066181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95781f-c25e-4b7c-bb2a-93a561e46a67 None admin] RBAC: Authorizing `identity:get_user(user_id=22aa11f7dfd046698081a55635933992)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:32.068026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95781f-c25e-4b7c-bb2a-93a561e46a67 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:32.073458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 130/250] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:31 2022] GET /identity/v3/users/22aa11f7dfd046698081a55635933992 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:32.075125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95781f-c25e-4b7c-bb2a-93a561e46a67 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:32.166664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f86eeb-07c5-43c3-836d-45cc0b3345c7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:32.178952 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95781f-c25e-4b7c-bb2a-93a561e46a67 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:32.208692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50a161-9c9e-4542-a3bd-a29a546ffb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:32.209651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50a161-9c9e-4542-a3bd-a29a546ffb0a None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.210271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50a161-9c9e-4542-a3bd-a29a546ffb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.210716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50a161-9c9e-4542-a3bd-a29a546ffb0a None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.211423 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619f7bb-8160-4a26-a17c-cf9a2877c84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:32.211750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50a161-9c9e-4542-a3bd-a29a546ffb0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:32.212289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619f7bb-8160-4a26-a17c-cf9a2877c84f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.212625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619f7bb-8160-4a26-a17c-cf9a2877c84f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.212899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619f7bb-8160-4a26-a17c-cf9a2877c84f None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.214106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619f7bb-8160-4a26-a17c-cf9a2877c84f None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:32.214493 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50a161-9c9e-4542-a3bd-a29a546ffb0a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:32.215549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619f7bb-8160-4a26-a17c-cf9a2877c84f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:32.217373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 131/251] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:32 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:32.222867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619f7bb-8160-4a26-a17c-cf9a2877c84f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:32.248772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 121/252] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:29:32 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:32.248772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50a161-9c9e-4542-a3bd-a29a546ffb0a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:32.339771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619f7bb-8160-4a26-a17c-cf9a2877c84f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:32.361493 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50a161-9c9e-4542-a3bd-a29a546ffb0a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:32.375868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da31a8-f43c-4d41-9ee3-6767b4783f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:32.376568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da31a8-f43c-4d41-9ee3-6767b4783f60 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.376913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da31a8-f43c-4d41-9ee3-6767b4783f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.377216 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da31a8-f43c-4d41-9ee3-6767b4783f60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.401378 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cca382-efa3-435c-9ef5-a200ac087d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:32.402202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cca382-efa3-435c-9ef5-a200ac087d85 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.402485 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cca382-efa3-435c-9ef5-a200ac087d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.402759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cca382-efa3-435c-9ef5-a200ac087d85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.414260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cca382-efa3-435c-9ef5-a200ac087d85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-54cca382-efa3-435c-9ef5-a200ac087d85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:32.422237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:32.423408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 122/253] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:32.429514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cca382-efa3-435c-9ef5-a200ac087d85 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:32.456318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da31a8-f43c-4d41-9ee3-6767b4783f60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:32.457992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da31a8-f43c-4d41-9ee3-6767b4783f60 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:32.537271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cca382-efa3-435c-9ef5-a200ac087d85 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:32.569513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 132/254] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:32.571503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cd862-611d-411b-8528-66d075cb0963 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.571788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cd862-611d-411b-8528-66d075cb0963 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.572277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cd862-611d-411b-8528-66d075cb0963 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.574175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 133/255] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:32.575034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cd862-611d-411b-8528-66d075cb0963 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:32.578864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9e055-945f-4145-ab29-184a5cb7c52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:32.579608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9e055-945f-4145-ab29-184a5cb7c52d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.579810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9e055-945f-4145-ab29-184a5cb7c52d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.580080 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9e055-945f-4145-ab29-184a5cb7c52d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.581010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9e055-945f-4145-ab29-184a5cb7c52d None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=24cf0228dd084700bf349d5310285eb0, user.id=3aae69207ed9492db04a8c0333864a63) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:32.581318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9e055-945f-4145-ab29-184a5cb7c52d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:32.582783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9e055-945f-4145-ab29-184a5cb7c52d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:32.590324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 123/256] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:32 2022] GET /identity/v3/role_assignments?user.id=3aae69207ed9492db04a8c0333864a63&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=e55ef11844a742f5889243f189bd59bc => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:32.591161 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9e055-945f-4145-ab29-184a5cb7c52d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:32.693674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cd862-611d-411b-8528-66d075cb0963 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:32.716847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9e055-945f-4145-ab29-184a5cb7c52d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:32.728817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3183ae-0a76-4f05-b9c7-b4139cefc7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:32.729504 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3183ae-0a76-4f05-b9c7-b4139cefc7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.729855 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3183ae-0a76-4f05-b9c7-b4139cefc7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.730048 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3183ae-0a76-4f05-b9c7-b4139cefc7b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.730795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3183ae-0a76-4f05-b9c7-b4139cefc7b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:32.731097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3183ae-0a76-4f05-b9c7-b4139cefc7b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:32.732508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3183ae-0a76-4f05-b9c7-b4139cefc7b3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:32.746069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 134/257] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:32.747651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f0ab0-a77b-4c5c-93b9-ea39856fba22 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.747918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f0ab0-a77b-4c5c-93b9-ea39856fba22 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.748176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f0ab0-a77b-4c5c-93b9-ea39856fba22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.754790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c404914-3739-42d6-9ea3-205ee2d36ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:32.756046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c404914-3739-42d6-9ea3-205ee2d36ecc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.756455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c404914-3739-42d6-9ea3-205ee2d36ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.756877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c404914-3739-42d6-9ea3-205ee2d36ecc None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.758317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c404914-3739-42d6-9ea3-205ee2d36ecc None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:32.760133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c404914-3739-42d6-9ea3-205ee2d36ecc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:32.762605 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 124/258] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:32 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:32.763572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c404914-3739-42d6-9ea3-205ee2d36ecc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:32.771049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ae0f0ab0-a77b-4c5c-93b9-ea39856fba22 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:32.772251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ae0f0ab0-a77b-4c5c-93b9-ea39856fba22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:32.866520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c404914-3739-42d6-9ea3-205ee2d36ecc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:32.893759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 135/259] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:32.896215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f0ab0-a77b-4c5c-93b9-ea39856fba22 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:32.919714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd4690-973e-4201-ab87-758dce0a4978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:32.920936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd4690-973e-4201-ab87-758dce0a4978 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:32.921376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd4690-973e-4201-ab87-758dce0a4978 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:32.921780 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd4690-973e-4201-ab87-758dce0a4978 None admin] PATH_INFO: `/v3/users/9c033a34517942fd918cb10e1f210dc2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:32.929319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd4690-973e-4201-ab87-758dce0a4978 None admin] RBAC: Authorizing `identity:get_user(user_id=9c033a34517942fd918cb10e1f210dc2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:32.930966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd4690-973e-4201-ab87-758dce0a4978 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:32.933065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 125/260] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:32 2022] GET /identity/v3/users/9c033a34517942fd918cb10e1f210dc2 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:32.938457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd4690-973e-4201-ab87-758dce0a4978 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:33.017659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f0ab0-a77b-4c5c-93b9-ea39856fba22 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:33.038218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd4690-973e-4201-ab87-758dce0a4978 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:33.054120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea60a8f-2288-4059-acd2-6f946a8dd94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:33.054801 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea60a8f-2288-4059-acd2-6f946a8dd94b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:33.055344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea60a8f-2288-4059-acd2-6f946a8dd94b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:33.055633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea60a8f-2288-4059-acd2-6f946a8dd94b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:33.056545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea60a8f-2288-4059-acd2-6f946a8dd94b None admin] RBAC: Adding query filter params (role.id=c6d29efc5be34cc097e61371c95cd446, scope.project.id=24cf0228dd084700bf349d5310285eb0, user.id=22aa11f7dfd046698081a55635933992) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:33.056857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea60a8f-2288-4059-acd2-6f946a8dd94b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:33.058323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea60a8f-2288-4059-acd2-6f946a8dd94b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:33.064651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 136/261] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:32 2022] GET /identity/v3/role_assignments?user.id=22aa11f7dfd046698081a55635933992&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=c6d29efc5be34cc097e61371c95cd446 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:33.069370 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea60a8f-2288-4059-acd2-6f946a8dd94b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:33.094314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997ac52-b879-4ec7-939c-c95843d79908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:33.095415 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997ac52-b879-4ec7-939c-c95843d79908 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:33.095857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997ac52-b879-4ec7-939c-c95843d79908 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:33.096261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997ac52-b879-4ec7-939c-c95843d79908 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:33.119742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997ac52-b879-4ec7-939c-c95843d79908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5997ac52-b879-4ec7-939c-c95843d79908 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:33.133082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:33.134157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:33.135145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:33.139718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 126/262] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:33.150164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997ac52-b879-4ec7-939c-c95843d79908 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:33.272244 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea60a8f-2288-4059-acd2-6f946a8dd94b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:33.340721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8d444-9588-48e5-88a7-f99835bfb20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:33.340721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8d444-9588-48e5-88a7-f99835bfb20b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:33.340721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8d444-9588-48e5-88a7-f99835bfb20b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:33.340721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8d444-9588-48e5-88a7-f99835bfb20b None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:33.341611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8d444-9588-48e5-88a7-f99835bfb20b None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:33.343085 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8d444-9588-48e5-88a7-f99835bfb20b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:33.345649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 137/263] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:33 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:33.346725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997ac52-b879-4ec7-939c-c95843d79908 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:33.352782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8d444-9588-48e5-88a7-f99835bfb20b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:33.383801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245e42c-8a31-4fc6-9ad5-a2d85fc60809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:33.385184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245e42c-8a31-4fc6-9ad5-a2d85fc60809 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:33.385693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245e42c-8a31-4fc6-9ad5-a2d85fc60809 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:33.386175 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245e42c-8a31-4fc6-9ad5-a2d85fc60809 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:33.387112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245e42c-8a31-4fc6-9ad5-a2d85fc60809 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:33.387600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245e42c-8a31-4fc6-9ad5-a2d85fc60809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:33.390241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245e42c-8a31-4fc6-9ad5-a2d85fc60809 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:33.399498 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 127/264] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:33.405774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245e42c-8a31-4fc6-9ad5-a2d85fc60809 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:33.455316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8d444-9588-48e5-88a7-f99835bfb20b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:33.488658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d660253-7057-485c-bb11-a98685b29b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:33.489561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d660253-7057-485c-bb11-a98685b29b53 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:33.489839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d660253-7057-485c-bb11-a98685b29b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:33.490109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d660253-7057-485c-bb11-a98685b29b53 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0/users/9c033a34517942fd918cb10e1f210dc2/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:33.492153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d660253-7057-485c-bb11-a98685b29b53 None admin] RBAC: Authorizing `identity:create_grant(project_id=24cf0228dd084700bf349d5310285eb0, user_id=9c033a34517942fd918cb10e1f210dc2, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:33.493533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d660253-7057-485c-bb11-a98685b29b53 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:33.516688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245e42c-8a31-4fc6-9ad5-a2d85fc60809 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:33.517867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 138/265] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:29:33 2022] PUT /identity/v3/projects/24cf0228dd084700bf349d5310285eb0/users/9c033a34517942fd918cb10e1f210dc2/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:29:33.553874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79e187-e848-43de-b836-173f1cb9c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:33.554772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79e187-e848-43de-b836-173f1cb9c3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:33.555050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79e187-e848-43de-b836-173f1cb9c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:33.555374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79e187-e848-43de-b836-173f1cb9c3e7 None admin] PATH_INFO: `/v3/users/3aae69207ed9492db04a8c0333864a63` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:33.556791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79e187-e848-43de-b836-173f1cb9c3e7 None admin] RBAC: Authorizing `identity:get_user(user_id=3aae69207ed9492db04a8c0333864a63)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:33.558335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79e187-e848-43de-b836-173f1cb9c3e7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:33.560043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 128/266] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:33 2022] GET /identity/v3/users/3aae69207ed9492db04a8c0333864a63 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:33.566281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d660253-7057-485c-bb11-a98685b29b53 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:33.672638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d660253-7057-485c-bb11-a98685b29b53 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:33.707822 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9e9dd-6c78-44b6-b9f8-81cedfc1ef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:33.708499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9e9dd-6c78-44b6-b9f8-81cedfc1ef91 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:33.708814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9e9dd-6c78-44b6-b9f8-81cedfc1ef91 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:33.709086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9e9dd-6c78-44b6-b9f8-81cedfc1ef91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:33.780976 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9e9dd-6c78-44b6-b9f8-81cedfc1ef91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:33.782469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9e9dd-6c78-44b6-b9f8-81cedfc1ef91 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:33.886141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 139/267] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:33.889496 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79e187-e848-43de-b836-173f1cb9c3e7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:33.987454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79e187-e848-43de-b836-173f1cb9c3e7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:34.020048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeff5d-e325-470f-8fd3-f1214d6e5d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:34.020951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeff5d-e325-470f-8fd3-f1214d6e5d3d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:34.021231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeff5d-e325-470f-8fd3-f1214d6e5d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:34.021495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeff5d-e325-470f-8fd3-f1214d6e5d3d None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:34.022679 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aeff5d-e325-470f-8fd3-f1214d6e5d3d None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:34.024137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aeff5d-e325-470f-8fd3-f1214d6e5d3d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:34.026277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 129/268] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:33 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:34.033623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9e9dd-6c78-44b6-b9f8-81cedfc1ef91 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:34.151124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9e9dd-6c78-44b6-b9f8-81cedfc1ef91 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:34.183789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65878185-d227-4d18-9613-5446d7da96bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:34.184461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65878185-d227-4d18-9613-5446d7da96bb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:34.184782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65878185-d227-4d18-9613-5446d7da96bb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:34.185108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65878185-d227-4d18-9613-5446d7da96bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:34.185944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65878185-d227-4d18-9613-5446d7da96bb None admin] RBAC: Adding query filter params (role.id=ec08a86158a44ee2b82951b44b228df2, scope.project.id=24cf0228dd084700bf349d5310285eb0, user.id=3aae69207ed9492db04a8c0333864a63) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:34.186252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65878185-d227-4d18-9613-5446d7da96bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:34.187714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65878185-d227-4d18-9613-5446d7da96bb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:34.193907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 140/269] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:34 2022] GET /identity/v3/role_assignments?user.id=3aae69207ed9492db04a8c0333864a63&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=ec08a86158a44ee2b82951b44b228df2 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:34.909261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae0e49-39d4-4d2c-8c0c-4aa02f9bb862 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:34.909760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae0e49-39d4-4d2c-8c0c-4aa02f9bb862 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:34.909760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae0e49-39d4-4d2c-8c0c-4aa02f9bb862 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:34.911239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 130/270] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:34.917302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb8efe-f630-4b8b-893c-3abe39f19928 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:34.917633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb8efe-f630-4b8b-893c-3abe39f19928 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:34.917862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb8efe-f630-4b8b-893c-3abe39f19928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:34.936926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-efeb8efe-f630-4b8b-893c-3abe39f19928 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:34.938164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-efeb8efe-f630-4b8b-893c-3abe39f19928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:35.005785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 141/271] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:35.029076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae0e49-39d4-4d2c-8c0c-4aa02f9bb862 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:35.131833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae0e49-39d4-4d2c-8c0c-4aa02f9bb862 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:35.137221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157abc4a-f3d3-4e2c-8c48-e8ac92b8e8d1 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:35.137721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157abc4a-f3d3-4e2c-8c48-e8ac92b8e8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:35.137796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157abc4a-f3d3-4e2c-8c48-e8ac92b8e8d1 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:35.139408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 142/272] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:29:35.145633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051bd6b-cda8-4059-ba40-eedcb511c44a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:35.146029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051bd6b-cda8-4059-ba40-eedcb511c44a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:35.146405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051bd6b-cda8-4059-ba40-eedcb511c44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:35.163002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e051bd6b-cda8-4059-ba40-eedcb511c44a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:35.164217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e051bd6b-cda8-4059-ba40-eedcb511c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:35.169692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665c891-b836-4705-8b53-52a06ac4dcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:35.170500 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665c891-b836-4705-8b53-52a06ac4dcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:35.170781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665c891-b836-4705-8b53-52a06ac4dcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:35.171051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665c891-b836-4705-8b53-52a06ac4dcb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:35.181216 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665c891-b836-4705-8b53-52a06ac4dcb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e665c891-b836-4705-8b53-52a06ac4dcb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:35.189070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:35.191109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:35.192269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 131/273] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:35.196793 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665c891-b836-4705-8b53-52a06ac4dcb8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:35.234810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 143/274] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:35.258950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051bd6b-cda8-4059-ba40-eedcb511c44a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:35.321412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665c891-b836-4705-8b53-52a06ac4dcb8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:35.356958 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051bd6b-cda8-4059-ba40-eedcb511c44a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:35.364202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f51324-da38-4605-8e3a-43ba501ee47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:35.365315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f51324-da38-4605-8e3a-43ba501ee47d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:35.365599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f51324-da38-4605-8e3a-43ba501ee47d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:35.365877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f51324-da38-4605-8e3a-43ba501ee47d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:35.367003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f51324-da38-4605-8e3a-43ba501ee47d None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:35.367315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f51324-da38-4605-8e3a-43ba501ee47d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:35.369463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f51324-da38-4605-8e3a-43ba501ee47d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:35.380166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 132/275] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:35.385810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f51324-da38-4605-8e3a-43ba501ee47d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:35.397555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46898a2-2f20-4eda-bf41-9cabeb5df413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:35.398707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46898a2-2f20-4eda-bf41-9cabeb5df413 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:35.399263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46898a2-2f20-4eda-bf41-9cabeb5df413 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:35.399545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46898a2-2f20-4eda-bf41-9cabeb5df413 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:35.401276 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46898a2-2f20-4eda-bf41-9cabeb5df413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:35.404190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46898a2-2f20-4eda-bf41-9cabeb5df413 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:35.406937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 144/276] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:35.413791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46898a2-2f20-4eda-bf41-9cabeb5df413 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:35.490269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f51324-da38-4605-8e3a-43ba501ee47d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:35.532334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edae1ac-cbc7-4e1b-8cca-180827bebd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:35.533253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edae1ac-cbc7-4e1b-8cca-180827bebd61 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:35.533531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edae1ac-cbc7-4e1b-8cca-180827bebd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:35.533796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edae1ac-cbc7-4e1b-8cca-180827bebd61 None admin] PATH_INFO: `/v3/users/5d9ec00b8180478abf1af0c71592a3cc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:35.543928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46898a2-2f20-4eda-bf41-9cabeb5df413 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:35.549246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edae1ac-cbc7-4e1b-8cca-180827bebd61 None admin] RBAC: Authorizing `identity:get_user(user_id=5d9ec00b8180478abf1af0c71592a3cc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:35.550725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edae1ac-cbc7-4e1b-8cca-180827bebd61 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:35.552620 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 133/277] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:35 2022] GET /identity/v3/users/5d9ec00b8180478abf1af0c71592a3cc => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:35.558771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edae1ac-cbc7-4e1b-8cca-180827bebd61 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:35.586806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55809821-14b8-4efe-b390-f96a35f8e9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:35.588200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55809821-14b8-4efe-b390-f96a35f8e9a4 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:35.588825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55809821-14b8-4efe-b390-f96a35f8e9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:35.589608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55809821-14b8-4efe-b390-f96a35f8e9a4 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:35.590828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55809821-14b8-4efe-b390-f96a35f8e9a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:35.593855 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55809821-14b8-4efe-b390-f96a35f8e9a4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:35.621481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 145/278] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:29:35 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:35.676879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edae1ac-cbc7-4e1b-8cca-180827bebd61 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:35.720740 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a58e2-f911-4b88-adbc-34ebf9b968e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:35.721888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a58e2-f911-4b88-adbc-34ebf9b968e1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:35.722387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a58e2-f911-4b88-adbc-34ebf9b968e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:35.722849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a58e2-f911-4b88-adbc-34ebf9b968e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:35.723890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a58e2-f911-4b88-adbc-34ebf9b968e1 None admin] RBAC: Adding query filter params (role.id=3385b41b990e4c6b88d77260a7822c7e, user.id=5d9ec00b8180478abf1af0c71592a3cc, scope.system=all) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:35.724398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a58e2-f911-4b88-adbc-34ebf9b968e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:35.726213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a58e2-f911-4b88-adbc-34ebf9b968e1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:35.733006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 134/279] 10.222.0.50 () {56 vars in 1308 bytes} [Fri Apr 29 18:29:35 2022] GET /identity/v3/role_assignments?user.id=5d9ec00b8180478abf1af0c71592a3cc&scope.system=all&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:35.887563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5edd59-cc32-4449-a5b1-b8b2c6aa3734 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:35.887907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5edd59-cc32-4449-a5b1-b8b2c6aa3734 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:35.888116 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5edd59-cc32-4449-a5b1-b8b2c6aa3734 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:35.889841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 146/280] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:35.895691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3e3a2-6ee4-4969-9c34-3b411643d611 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:35.896221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3e3a2-6ee4-4969-9c34-3b411643d611 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:35.896759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3e3a2-6ee4-4969-9c34-3b411643d611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:35.917719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-61c3e3a2-6ee4-4969-9c34-3b411643d611 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:35.919592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-61c3e3a2-6ee4-4969-9c34-3b411643d611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:35.989784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 135/281] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:36.011777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5edd59-cc32-4449-a5b1-b8b2c6aa3734 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:36.141097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5edd59-cc32-4449-a5b1-b8b2c6aa3734 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:36.186221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfb274-0a84-4ad8-ac64-87a39c8cfd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:36.187089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfb274-0a84-4ad8-ac64-87a39c8cfd02 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:36.187375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfb274-0a84-4ad8-ac64-87a39c8cfd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:36.187648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfb274-0a84-4ad8-ac64-87a39c8cfd02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:36.201248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfb274-0a84-4ad8-ac64-87a39c8cfd02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-c6bfb274-0a84-4ad8-ac64-87a39c8cfd02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:36.209083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:36.209984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:36.210773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 147/282] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:36.216334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3e3a2-6ee4-4969-9c34-3b411643d611 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:36.246948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e2097-44c7-4c9f-abdf-c5e73404e885 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:36.247257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e2097-44c7-4c9f-abdf-c5e73404e885 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:36.247419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e2097-44c7-4c9f-abdf-c5e73404e885 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:36.249017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 148/283] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:36.253922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d757f8-67e3-4db6-87d5-847613fb4511 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:36.254205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d757f8-67e3-4db6-87d5-847613fb4511 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:36.260615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d757f8-67e3-4db6-87d5-847613fb4511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:36.274079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-61d757f8-67e3-4db6-87d5-847613fb4511 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:36.276230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-61d757f8-67e3-4db6-87d5-847613fb4511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:36.379631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3e3a2-6ee4-4969-9c34-3b411643d611 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:36.413389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643e5f1-cacb-4c0f-90ec-11b3002ad8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:36.414060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643e5f1-cacb-4c0f-90ec-11b3002ad8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:36.414341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643e5f1-cacb-4c0f-90ec-11b3002ad8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:36.414610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643e5f1-cacb-4c0f-90ec-11b3002ad8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:36.415347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643e5f1-cacb-4c0f-90ec-11b3002ad8a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:36.415741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643e5f1-cacb-4c0f-90ec-11b3002ad8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:36.417256 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 149/284] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:36.417751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643e5f1-cacb-4c0f-90ec-11b3002ad8a2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:36.432678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 136/285] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:36.438148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d757f8-67e3-4db6-87d5-847613fb4511 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:36.456522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643e5f1-cacb-4c0f-90ec-11b3002ad8a2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:36.552645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643e5f1-cacb-4c0f-90ec-11b3002ad8a2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:36.587003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b70c03-f286-4086-bd2f-9469f100860b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:36.588467 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b70c03-f286-4086-bd2f-9469f100860b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:36.588788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b70c03-f286-4086-bd2f-9469f100860b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:36.589064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b70c03-f286-4086-bd2f-9469f100860b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:36.599596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b70c03-f286-4086-bd2f-9469f100860b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-15b70c03-f286-4086-bd2f-9469f100860b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:36.608798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:36.609713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:36.610506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 137/286] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:36.616025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b70c03-f286-4086-bd2f-9469f100860b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:36.674629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d757f8-67e3-4db6-87d5-847613fb4511 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:36.718764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b70c03-f286-4086-bd2f-9469f100860b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:36.753556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e51ba3-af52-49d0-8450-fa7a6dc14640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:36.754650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e51ba3-af52-49d0-8450-fa7a6dc14640 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:36.755016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e51ba3-af52-49d0-8450-fa7a6dc14640 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:36.755382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e51ba3-af52-49d0-8450-fa7a6dc14640 None admin] PATH_INFO: `/v3/users/22aa11f7dfd046698081a55635933992` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:36.755922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89203906-9e2f-4581-b726-4f53ccff7731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:36.756616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e51ba3-af52-49d0-8450-fa7a6dc14640 None admin] RBAC: Authorizing `identity:get_user(user_id=22aa11f7dfd046698081a55635933992)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:36.756722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89203906-9e2f-4581-b726-4f53ccff7731 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:36.756909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89203906-9e2f-4581-b726-4f53ccff7731 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:36.757182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89203906-9e2f-4581-b726-4f53ccff7731 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:36.757935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89203906-9e2f-4581-b726-4f53ccff7731 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:36.758052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e51ba3-af52-49d0-8450-fa7a6dc14640 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:36.758239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89203906-9e2f-4581-b726-4f53ccff7731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:36.759749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89203906-9e2f-4581-b726-4f53ccff7731 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:36.760006 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 150/287] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:36 2022] GET /identity/v3/users/22aa11f7dfd046698081a55635933992 => generated 297 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:36.766373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e51ba3-af52-49d0-8450-fa7a6dc14640 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:36.769806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 138/288] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:36.777154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89203906-9e2f-4581-b726-4f53ccff7731 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:36.880816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89203906-9e2f-4581-b726-4f53ccff7731 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:36.913828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e51ba3-af52-49d0-8450-fa7a6dc14640 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:36.916296 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec794575-1896-40ba-a214-df5fd670fb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:36.917698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec794575-1896-40ba-a214-df5fd670fb0c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:36.918038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec794575-1896-40ba-a214-df5fd670fb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:36.918564 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec794575-1896-40ba-a214-df5fd670fb0c None admin] PATH_INFO: `/v3/users/9c033a34517942fd918cb10e1f210dc2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:36.919580 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec794575-1896-40ba-a214-df5fd670fb0c None admin] RBAC: Authorizing `identity:get_user(user_id=9c033a34517942fd918cb10e1f210dc2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:36.921111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec794575-1896-40ba-a214-df5fd670fb0c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:36.923049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 139/289] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:36 2022] GET /identity/v3/users/9c033a34517942fd918cb10e1f210dc2 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:36.928537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec794575-1896-40ba-a214-df5fd670fb0c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:36.947543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224bccf-9d6b-4861-9a7e-93b7e393a8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:36.948418 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224bccf-9d6b-4861-9a7e-93b7e393a8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:36.948719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224bccf-9d6b-4861-9a7e-93b7e393a8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:36.948985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224bccf-9d6b-4861-9a7e-93b7e393a8b0 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:36.950164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224bccf-9d6b-4861-9a7e-93b7e393a8b0 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:36.951603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224bccf-9d6b-4861-9a7e-93b7e393a8b0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:36.953320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 151/290] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:36 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:36.961386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224bccf-9d6b-4861-9a7e-93b7e393a8b0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:37.032463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec794575-1896-40ba-a214-df5fd670fb0c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:37.067428 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede78589-d3fc-44c9-b87a-fffae44104fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:37.068079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede78589-d3fc-44c9-b87a-fffae44104fa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:37.068361 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede78589-d3fc-44c9-b87a-fffae44104fa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:37.068674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede78589-d3fc-44c9-b87a-fffae44104fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:37.077312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224bccf-9d6b-4861-9a7e-93b7e393a8b0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:37.110323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171a305-469f-4391-8625-702df0fb003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:37.111226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171a305-469f-4391-8625-702df0fb003a None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:37.111502 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171a305-469f-4391-8625-702df0fb003a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:37.111767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171a305-469f-4391-8625-702df0fb003a None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0/users/22aa11f7dfd046698081a55635933992/roles/c6d29efc5be34cc097e61371c95cd446` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:37.113741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171a305-469f-4391-8625-702df0fb003a None admin] RBAC: Authorizing `identity:create_grant(project_id=24cf0228dd084700bf349d5310285eb0, user_id=22aa11f7dfd046698081a55635933992, role_id=c6d29efc5be34cc097e61371c95cd446)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:37.115193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171a305-469f-4391-8625-702df0fb003a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:37.141792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede78589-d3fc-44c9-b87a-fffae44104fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:37.143011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 152/291] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:29:36 2022] PUT /identity/v3/projects/24cf0228dd084700bf349d5310285eb0/users/22aa11f7dfd046698081a55635933992/roles/c6d29efc5be34cc097e61371c95cd446 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:37.143537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede78589-d3fc-44c9-b87a-fffae44104fa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:37.256310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 140/292] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Apr 29 18:29:37.263254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171a305-469f-4391-8625-702df0fb003a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:37.396452 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171a305-469f-4391-8625-702df0fb003a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:37.433032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d1e7a-71f7-44cb-aeb4-af64673f1b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:37.434262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d1e7a-71f7-44cb-aeb4-af64673f1b51 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:37.434714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d1e7a-71f7-44cb-aeb4-af64673f1b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:37.435153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d1e7a-71f7-44cb-aeb4-af64673f1b51 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:37.436574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d1e7a-71f7-44cb-aeb4-af64673f1b51 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:37.438205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d1e7a-71f7-44cb-aeb4-af64673f1b51 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:37.440215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 153/293] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:37 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:37.446325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede78589-d3fc-44c9-b87a-fffae44104fa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:37.548482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede78589-d3fc-44c9-b87a-fffae44104fa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:37.583420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8af73b-2467-483b-bc60-b12a34f686f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:37.584403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8af73b-2467-483b-bc60-b12a34f686f8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:37.584878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8af73b-2467-483b-bc60-b12a34f686f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:37.585313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8af73b-2467-483b-bc60-b12a34f686f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:37.586322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8af73b-2467-483b-bc60-b12a34f686f8 None admin] RBAC: Adding query filter params (role.id=3385b41b990e4c6b88d77260a7822c7e, scope.project.id=24cf0228dd084700bf349d5310285eb0, user.id=9c033a34517942fd918cb10e1f210dc2) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:37.586801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8af73b-2467-483b-bc60-b12a34f686f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:37.589058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8af73b-2467-483b-bc60-b12a34f686f8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:37.596105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 141/294] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:37 2022] GET /identity/v3/role_assignments?user.id=9c033a34517942fd918cb10e1f210dc2&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:38.394957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c434c5-205a-431b-8d09-6269cc8b5795 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:38.395526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c434c5-205a-431b-8d09-6269cc8b5795 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:38.395526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c434c5-205a-431b-8d09-6269cc8b5795 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:38.397537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 154/295] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:38.404769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f29b9a-4cf8-47c7-bcd0-315e87123be6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:38.405320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f29b9a-4cf8-47c7-bcd0-315e87123be6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:38.405320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f29b9a-4cf8-47c7-bcd0-315e87123be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:38.425451 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-34f29b9a-4cf8-47c7-bcd0-315e87123be6 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:38.427067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34f29b9a-4cf8-47c7-bcd0-315e87123be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:38.499117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 142/296] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:38.521090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c434c5-205a-431b-8d09-6269cc8b5795 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:38.596776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601e23a-f09c-4d3a-8c5f-2927afe6b60e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:38.597137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601e23a-f09c-4d3a-8c5f-2927afe6b60e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:38.597332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601e23a-f09c-4d3a-8c5f-2927afe6b60e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:38.600018 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 143/297] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:29:38.604563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b3ed7-598a-4784-9d73-0f3b8c078ea4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:38.604860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b3ed7-598a-4784-9d73-0f3b8c078ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:38.605140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b3ed7-598a-4784-9d73-0f3b8c078ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:38.621856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-748b3ed7-598a-4784-9d73-0f3b8c078ea4 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:38.623508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-748b3ed7-598a-4784-9d73-0f3b8c078ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:38.631941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c434c5-205a-431b-8d09-6269cc8b5795 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:38.664535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17acbcc8-5ffb-4374-b426-ce021fd04823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:38.665254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acbcc8-5ffb-4374-b426-ce021fd04823 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:38.665592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acbcc8-5ffb-4374-b426-ce021fd04823 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:38.665867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acbcc8-5ffb-4374-b426-ce021fd04823 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:38.675349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17acbcc8-5ffb-4374-b426-ce021fd04823 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-17acbcc8-5ffb-4374-b426-ce021fd04823 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:38.682969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:38.683844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:38.685032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 155/298] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:38.685858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666886c7-83ad-46c2-a413-29fa867bc160 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:38.686114 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666886c7-83ad-46c2-a413-29fa867bc160 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:38.686387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666886c7-83ad-46c2-a413-29fa867bc160 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:38.688527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 156/299] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:38.692334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666886c7-83ad-46c2-a413-29fa867bc160 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:38.695157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 144/300] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:38.696306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d352607-1637-4237-8c0a-095271e5ac35 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:38.696640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d352607-1637-4237-8c0a-095271e5ac35 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:38.696990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d352607-1637-4237-8c0a-095271e5ac35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:38.719242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2d352607-1637-4237-8c0a-095271e5ac35 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:38.720913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2d352607-1637-4237-8c0a-095271e5ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:38.787119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 145/301] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:38.788297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d352607-1637-4237-8c0a-095271e5ac35 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:38.803322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666886c7-83ad-46c2-a413-29fa867bc160 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:38.836202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054f028-b726-43cc-8b84-73685507cb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:38.836985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054f028-b726-43cc-8b84-73685507cb92 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:38.837266 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054f028-b726-43cc-8b84-73685507cb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:38.837568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054f028-b726-43cc-8b84-73685507cb92 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:38.838316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054f028-b726-43cc-8b84-73685507cb92 None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:38.838612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054f028-b726-43cc-8b84-73685507cb92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:38.839988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054f028-b726-43cc-8b84-73685507cb92 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:38.848309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 157/302] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:38.849132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054f028-b726-43cc-8b84-73685507cb92 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:38.898169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d352607-1637-4237-8c0a-095271e5ac35 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:38.932202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569ce44-6271-4170-bca8-f1105f2fdec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:38.932955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ce44-6271-4170-bca8-f1105f2fdec4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:38.933237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ce44-6271-4170-bca8-f1105f2fdec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:38.933507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ce44-6271-4170-bca8-f1105f2fdec4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:38.944160 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569ce44-6271-4170-bca8-f1105f2fdec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:38.949084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054f028-b726-43cc-8b84-73685507cb92 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4569ce44-6271-4170-bca8-f1105f2fdec4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:38.952069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:38.953079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:38.954259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 146/303] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:38.954915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569ce44-6271-4170-bca8-f1105f2fdec4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:38.984655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde78e4-0c9c-468f-b6bf-14eef35c5f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:38.985463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde78e4-0c9c-468f-b6bf-14eef35c5f2f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:38.985742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde78e4-0c9c-468f-b6bf-14eef35c5f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:38.986007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde78e4-0c9c-468f-b6bf-14eef35c5f2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:38.987182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde78e4-0c9c-468f-b6bf-14eef35c5f2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:38.988654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde78e4-0c9c-468f-b6bf-14eef35c5f2f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:38.994913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 158/304] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:38.994913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde78e4-0c9c-468f-b6bf-14eef35c5f2f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:39.065290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569ce44-6271-4170-bca8-f1105f2fdec4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:39.090857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde78e4-0c9c-468f-b6bf-14eef35c5f2f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:39.098813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe408c-d7a3-47ab-9b74-8bea77686fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:39.099702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe408c-d7a3-47ab-9b74-8bea77686fcb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:39.099981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe408c-d7a3-47ab-9b74-8bea77686fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:39.100254 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe408c-d7a3-47ab-9b74-8bea77686fcb None admin] PATH_INFO: `/v3/users/5d9ec00b8180478abf1af0c71592a3cc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:39.101484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe408c-d7a3-47ab-9b74-8bea77686fcb None admin] RBAC: Authorizing `identity:get_user(user_id=5d9ec00b8180478abf1af0c71592a3cc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:39.102960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe408c-d7a3-47ab-9b74-8bea77686fcb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:39.109427 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 147/305] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:38 2022] GET /identity/v3/users/5d9ec00b8180478abf1af0c71592a3cc => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:39.109427 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe408c-d7a3-47ab-9b74-8bea77686fcb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:39.128986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74c479-b3ab-48bb-9684-639b3644b92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:39.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74c479-b3ab-48bb-9684-639b3644b92c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:39.129947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74c479-b3ab-48bb-9684-639b3644b92c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:39.130218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74c479-b3ab-48bb-9684-639b3644b92c None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:39.130951 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74c479-b3ab-48bb-9684-639b3644b92c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:39.131253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74c479-b3ab-48bb-9684-639b3644b92c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:39.132660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74c479-b3ab-48bb-9684-639b3644b92c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:39.141276 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 159/306] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:39.146484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74c479-b3ab-48bb-9684-639b3644b92c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:39.209683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe408c-d7a3-47ab-9b74-8bea77686fcb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:39.242008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ca43a-d708-4d48-8bd2-e2b09db0d830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:39.242676 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ca43a-d708-4d48-8bd2-e2b09db0d830 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:39.242793 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74c479-b3ab-48bb-9684-639b3644b92c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:39.242967 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ca43a-d708-4d48-8bd2-e2b09db0d830 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:39.243217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ca43a-d708-4d48-8bd2-e2b09db0d830 None admin] PATH_INFO: `/v3/groups` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:39.244935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ca43a-d708-4d48-8bd2-e2b09db0d830 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:39.246375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ca43a-d708-4d48-8bd2-e2b09db0d830 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:39.261741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 148/307] 10.222.0.50 () {60 vars in 1134 bytes} [Fri Apr 29 18:29:39 2022] POST /identity/v3/groups => generated 229 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:29:39.262869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ca43a-d708-4d48-8bd2-e2b09db0d830 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:39.293992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefe96a-71d1-4a47-a801-83260004c447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:39.294956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefe96a-71d1-4a47-a801-83260004c447 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:39.295296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefe96a-71d1-4a47-a801-83260004c447 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:39.295580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefe96a-71d1-4a47-a801-83260004c447 None admin] PATH_INFO: `/v3/system/users/5d9ec00b8180478abf1af0c71592a3cc/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:39.298318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefe96a-71d1-4a47-a801-83260004c447 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5d9ec00b8180478abf1af0c71592a3cc, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:39.300022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefe96a-71d1-4a47-a801-83260004c447 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:39.320560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 160/308] 10.222.0.50 () {58 vars in 1334 bytes} [Fri Apr 29 18:29:39 2022] PUT /identity/v3/system/users/5d9ec00b8180478abf1af0c71592a3cc/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:39.390402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ca43a-d708-4d48-8bd2-e2b09db0d830 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:39.430033 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46d89f-dfbe-4348-920b-9e62ae1cdace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:39.430949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46d89f-dfbe-4348-920b-9e62ae1cdace None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:39.431326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46d89f-dfbe-4348-920b-9e62ae1cdace None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:39.431654 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46d89f-dfbe-4348-920b-9e62ae1cdace None admin] PATH_INFO: `/v3/users/1c21d5d72b364361a4e68f22dd696429` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:39.448123 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46d89f-dfbe-4348-920b-9e62ae1cdace None admin] RBAC: Authorizing `identity:get_user(user_id=1c21d5d72b364361a4e68f22dd696429)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:39.449672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46d89f-dfbe-4348-920b-9e62ae1cdace None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:39.451943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 149/309] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:39 2022] GET /identity/v3/users/1c21d5d72b364361a4e68f22dd696429 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:39.457909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefe96a-71d1-4a47-a801-83260004c447 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:39.570161 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefe96a-71d1-4a47-a801-83260004c447 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:39.608946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4f23a-2220-45a2-817c-7331faf7a3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:39.609616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4f23a-2220-45a2-817c-7331faf7a3bb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:39.609899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4f23a-2220-45a2-817c-7331faf7a3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:39.610169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4f23a-2220-45a2-817c-7331faf7a3bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:39.610995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4f23a-2220-45a2-817c-7331faf7a3bb None admin] RBAC: Adding query filter params (role.id=c6d29efc5be34cc097e61371c95cd446, user.id=1c21d5d72b364361a4e68f22dd696429, scope.system=all) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:39.611300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4f23a-2220-45a2-817c-7331faf7a3bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:39.612779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4f23a-2220-45a2-817c-7331faf7a3bb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:39.618314 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 161/310] 10.222.0.50 () {56 vars in 1308 bytes} [Fri Apr 29 18:29:39 2022] GET /identity/v3/role_assignments?user.id=1c21d5d72b364361a4e68f22dd696429&scope.system=all&role.id=c6d29efc5be34cc097e61371c95cd446 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:39.741118 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0fc33-2419-4554-a87d-4b152b1f8562 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:39.741486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0fc33-2419-4554-a87d-4b152b1f8562 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:39.741673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0fc33-2419-4554-a87d-4b152b1f8562 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:39.743139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 150/311] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:39.752323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417af2d5-72ce-475b-ae5d-db5c5d0a2326 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:39.752682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417af2d5-72ce-475b-ae5d-db5c5d0a2326 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:39.752956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417af2d5-72ce-475b-ae5d-db5c5d0a2326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:39.772971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-417af2d5-72ce-475b-ae5d-db5c5d0a2326 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:39.774532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-417af2d5-72ce-475b-ae5d-db5c5d0a2326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:39.849583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 162/312] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:39.873914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0fc33-2419-4554-a87d-4b152b1f8562 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:39.977067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0fc33-2419-4554-a87d-4b152b1f8562 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:40.011678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224295-025b-4675-9cd0-3a0120bd24b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:40.012484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224295-025b-4675-9cd0-3a0120bd24b6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:40.012786 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224295-025b-4675-9cd0-3a0120bd24b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:40.013064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224295-025b-4675-9cd0-3a0120bd24b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:40.027360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18224295-025b-4675-9cd0-3a0120bd24b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-18224295-025b-4675-9cd0-3a0120bd24b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:40.035052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:40.035934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:40.036788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 151/313] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:40.044870 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417af2d5-72ce-475b-ae5d-db5c5d0a2326 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:40.142771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417af2d5-72ce-475b-ae5d-db5c5d0a2326 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:40.180180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f49c-ad98-4c83-8390-75f15041faa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:40.180912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f49c-ad98-4c83-8390-75f15041faa7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:40.181253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f49c-ad98-4c83-8390-75f15041faa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:40.181559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f49c-ad98-4c83-8390-75f15041faa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:40.182310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3f49c-ad98-4c83-8390-75f15041faa7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:40.182613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3f49c-ad98-4c83-8390-75f15041faa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:40.184027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3f49c-ad98-4c83-8390-75f15041faa7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:40.192332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 163/314] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:40.199484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224295-025b-4675-9cd0-3a0120bd24b6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:40.301469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224295-025b-4675-9cd0-3a0120bd24b6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:40.335189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606e685-ef0d-40ec-bbb5-7ff0e7124762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:40.336076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606e685-ef0d-40ec-bbb5-7ff0e7124762 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:40.336357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606e685-ef0d-40ec-bbb5-7ff0e7124762 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:40.336670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606e685-ef0d-40ec-bbb5-7ff0e7124762 None admin] PATH_INFO: `/v3/users/22aa11f7dfd046698081a55635933992` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:40.338468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606e685-ef0d-40ec-bbb5-7ff0e7124762 None admin] RBAC: Authorizing `identity:get_user(user_id=22aa11f7dfd046698081a55635933992)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:40.339975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606e685-ef0d-40ec-bbb5-7ff0e7124762 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:40.342020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 152/315] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:40 2022] GET /identity/v3/users/22aa11f7dfd046698081a55635933992 => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:40.347898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f49c-ad98-4c83-8390-75f15041faa7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:40.450304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f49c-ad98-4c83-8390-75f15041faa7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:40.486166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d70c4-3348-4d36-92b7-ed1a998f1bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:40.487500 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d70c4-3348-4d36-92b7-ed1a998f1bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:40.487928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d70c4-3348-4d36-92b7-ed1a998f1bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:40.488344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d70c4-3348-4d36-92b7-ed1a998f1bc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:40.556042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d70c4-3348-4d36-92b7-ed1a998f1bc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:40.557918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d70c4-3348-4d36-92b7-ed1a998f1bc3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:40.653688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 164/316] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:40.658973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606e685-ef0d-40ec-bbb5-7ff0e7124762 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:40.767685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606e685-ef0d-40ec-bbb5-7ff0e7124762 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:40.801254 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0268392-b52b-4077-be6d-e58a3fc99046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:40.802607 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0268392-b52b-4077-be6d-e58a3fc99046 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:40.803095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0268392-b52b-4077-be6d-e58a3fc99046 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:40.803560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0268392-b52b-4077-be6d-e58a3fc99046 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:40.805008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0268392-b52b-4077-be6d-e58a3fc99046 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:40.806823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0268392-b52b-4077-be6d-e58a3fc99046 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:40.809125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 153/317] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:40 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:40.815427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d70c4-3348-4d36-92b7-ed1a998f1bc3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:40.920836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d70c4-3348-4d36-92b7-ed1a998f1bc3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:40.954019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff98fa2-2451-438e-a8ef-502110e370db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:40.954675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff98fa2-2451-438e-a8ef-502110e370db None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:40.954948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff98fa2-2451-438e-a8ef-502110e370db None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:40.955213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff98fa2-2451-438e-a8ef-502110e370db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:40.956035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff98fa2-2451-438e-a8ef-502110e370db None admin] RBAC: Adding query filter params (role.id=c6d29efc5be34cc097e61371c95cd446, scope.project.id=24cf0228dd084700bf349d5310285eb0, user.id=22aa11f7dfd046698081a55635933992) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:40.956336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff98fa2-2451-438e-a8ef-502110e370db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:40.957803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff98fa2-2451-438e-a8ef-502110e370db None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:40.964112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 165/318] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:29:40 2022] GET /identity/v3/role_assignments?user.id=22aa11f7dfd046698081a55635933992&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=c6d29efc5be34cc097e61371c95cd446 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:41.629091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09d16-2e41-4689-84a8-95552a8dde56 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:41.629376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09d16-2e41-4689-84a8-95552a8dde56 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:41.629646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09d16-2e41-4689-84a8-95552a8dde56 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:41.631045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 154/319] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:41.636514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2eadf-3aed-4c70-9fd4-04c6937c62a2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:41.636986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2eadf-3aed-4c70-9fd4-04c6937c62a2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:41.637057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2eadf-3aed-4c70-9fd4-04c6937c62a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:41.653860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-29f2eadf-3aed-4c70-9fd4-04c6937c62a2 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:41.655034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-29f2eadf-3aed-4c70-9fd4-04c6937c62a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:41.685559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478ea49-cf27-4344-85e5-d5a53e877c16 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:41.686103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478ea49-cf27-4344-85e5-d5a53e877c16 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:41.686103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478ea49-cf27-4344-85e5-d5a53e877c16 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:41.687615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 155/320] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:29:41.694741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576bfe-2b03-4ff0-83be-fc228c0cf5d8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:41.695046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576bfe-2b03-4ff0-83be-fc228c0cf5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:41.695315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2576bfe-2b03-4ff0-83be-fc228c0cf5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:41.712325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e2576bfe-2b03-4ff0-83be-fc228c0cf5d8 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:41.713718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e2576bfe-2b03-4ff0-83be-fc228c0cf5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:41.725941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 166/321] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:41.754799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2eadf-3aed-4c70-9fd4-04c6937c62a2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:41.780688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 156/322] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:41.804019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2576bfe-2b03-4ff0-83be-fc228c0cf5d8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:41.851559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2eadf-3aed-4c70-9fd4-04c6937c62a2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:41.887349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ace2f-71f1-4c8c-a191-d93d8a5e30e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:41.888075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ace2f-71f1-4c8c-a191-d93d8a5e30e0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:41.888354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ace2f-71f1-4c8c-a191-d93d8a5e30e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:41.888658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ace2f-71f1-4c8c-a191-d93d8a5e30e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:41.889857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ace2f-71f1-4c8c-a191-d93d8a5e30e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:41.891419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ace2f-71f1-4c8c-a191-d93d8a5e30e0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:41.893264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 167/323] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:29:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:41.899095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ace2f-71f1-4c8c-a191-d93d8a5e30e0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:41.908908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2576bfe-2b03-4ff0-83be-fc228c0cf5d8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:41.941926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3fd1a-fb60-45f9-94e4-a8b157d87110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:41.942720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3fd1a-fb60-45f9-94e4-a8b157d87110 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:41.943010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3fd1a-fb60-45f9-94e4-a8b157d87110 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:41.943275 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3fd1a-fb60-45f9-94e4-a8b157d87110 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:41.953082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3fd1a-fb60-45f9-94e4-a8b157d87110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-fca3fd1a-fb60-45f9-94e4-a8b157d87110 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:41.960718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:41.961590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:41.962504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 157/324] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:41.967676 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3fd1a-fb60-45f9-94e4-a8b157d87110 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:41.997187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ace2f-71f1-4c8c-a191-d93d8a5e30e0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:42.030052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288306d8-9caf-4fd1-ae65-8c6d5063d772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:42.030728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288306d8-9caf-4fd1-ae65-8c6d5063d772 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:42.031004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288306d8-9caf-4fd1-ae65-8c6d5063d772 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:42.031559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288306d8-9caf-4fd1-ae65-8c6d5063d772 None admin] PATH_INFO: `/v3/groups` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:42.032243 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288306d8-9caf-4fd1-ae65-8c6d5063d772 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:42.033711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288306d8-9caf-4fd1-ae65-8c6d5063d772 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:42.047891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 168/325] 10.222.0.50 () {60 vars in 1134 bytes} [Fri Apr 29 18:29:41 2022] POST /identity/v3/groups => generated 226 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:29:42.049232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ffcd0-d89d-45ac-bf28-456b4cf669d3 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:42.049515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ffcd0-d89d-45ac-bf28-456b4cf669d3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:42.049783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ffcd0-d89d-45ac-bf28-456b4cf669d3 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:42.051003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 169/326] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:42.056967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c6166-cfbc-4afc-88f7-8c286463de3b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:42.057332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c6166-cfbc-4afc-88f7-8c286463de3b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:42.057558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c6166-cfbc-4afc-88f7-8c286463de3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:42.071456 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3fd1a-fb60-45f9-94e4-a8b157d87110 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:42.075873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-425c6166-cfbc-4afc-88f7-8c286463de3b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:42.077362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-425c6166-cfbc-4afc-88f7-8c286463de3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:42.108828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d5d6d-cada-434a-a9c6-658015ecddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:42.109518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d5d6d-cada-434a-a9c6-658015ecddb9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:42.109792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d5d6d-cada-434a-a9c6-658015ecddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:42.110063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d5d6d-cada-434a-a9c6-658015ecddb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:42.110823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d5d6d-cada-434a-a9c6-658015ecddb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:42.111146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d5d6d-cada-434a-a9c6-658015ecddb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:42.112603 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d5d6d-cada-434a-a9c6-658015ecddb9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:42.122672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 158/327] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:42.127991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d5d6d-cada-434a-a9c6-658015ecddb9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:42.152926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 170/328] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:42.175581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c6166-cfbc-4afc-88f7-8c286463de3b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:42.235055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d5d6d-cada-434a-a9c6-658015ecddb9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:42.268655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a723d-caad-45d6-b71a-15e18265c0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:42.269537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a723d-caad-45d6-b71a-15e18265c0dd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:42.269813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a723d-caad-45d6-b71a-15e18265c0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:42.270079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a723d-caad-45d6-b71a-15e18265c0dd None admin] PATH_INFO: `/v3/users/5d9ec00b8180478abf1af0c71592a3cc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:42.271256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a723d-caad-45d6-b71a-15e18265c0dd None admin] RBAC: Authorizing `identity:get_user(user_id=5d9ec00b8180478abf1af0c71592a3cc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:42.272740 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a723d-caad-45d6-b71a-15e18265c0dd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:42.274528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 159/329] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:42 2022] GET /identity/v3/users/5d9ec00b8180478abf1af0c71592a3cc => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:42.274633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c6166-cfbc-4afc-88f7-8c286463de3b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:42.280994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a723d-caad-45d6-b71a-15e18265c0dd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:42.316607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baaaf5f-995e-444f-9eaa-c00390de4085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:42.316607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaaf5f-995e-444f-9eaa-c00390de4085 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:42.316900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaaf5f-995e-444f-9eaa-c00390de4085 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:42.317608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaaf5f-995e-444f-9eaa-c00390de4085 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:42.329261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baaaf5f-995e-444f-9eaa-c00390de4085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2baaaf5f-995e-444f-9eaa-c00390de4085 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:42.337942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:42.339432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:42.340421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 171/330] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:42.345628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baaaf5f-995e-444f-9eaa-c00390de4085 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:42.385311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a723d-caad-45d6-b71a-15e18265c0dd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:42.421594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75825c8-3fa3-41ee-abfc-773258b8c026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:42.422492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75825c8-3fa3-41ee-abfc-773258b8c026 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:42.422933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75825c8-3fa3-41ee-abfc-773258b8c026 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:42.423351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75825c8-3fa3-41ee-abfc-773258b8c026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:42.424422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75825c8-3fa3-41ee-abfc-773258b8c026 None admin] RBAC: Adding query filter params (role.id=3385b41b990e4c6b88d77260a7822c7e, user.id=5d9ec00b8180478abf1af0c71592a3cc, scope.system=all) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:42.424890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75825c8-3fa3-41ee-abfc-773258b8c026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:42.426486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75825c8-3fa3-41ee-abfc-773258b8c026 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:42.439555 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 160/331] 10.222.0.50 () {56 vars in 1308 bytes} [Fri Apr 29 18:29:42 2022] GET /identity/v3/role_assignments?user.id=5d9ec00b8180478abf1af0c71592a3cc&scope.system=all&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 510 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:42.466645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baaaf5f-995e-444f-9eaa-c00390de4085 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:42.512041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f7950-a75d-4e0b-910c-566a0bcc0f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:42.512983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f7950-a75d-4e0b-910c-566a0bcc0f5b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:42.513363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f7950-a75d-4e0b-910c-566a0bcc0f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:42.513748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f7950-a75d-4e0b-910c-566a0bcc0f5b None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:42.514631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f7950-a75d-4e0b-910c-566a0bcc0f5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:42.515035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f7950-a75d-4e0b-910c-566a0bcc0f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:42.516666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f7950-a75d-4e0b-910c-566a0bcc0f5b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:42.528827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 172/332] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:42.534808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75825c8-3fa3-41ee-abfc-773258b8c026 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:42.650289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75825c8-3fa3-41ee-abfc-773258b8c026 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:42.685258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e66eb-77e4-40b4-97ac-258d876ab023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:42.686490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e66eb-77e4-40b4-97ac-258d876ab023 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:42.687400 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e66eb-77e4-40b4-97ac-258d876ab023 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:42.687812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e66eb-77e4-40b4-97ac-258d876ab023 None admin] PATH_INFO: `/v3/users/1c21d5d72b364361a4e68f22dd696429` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:42.689247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e66eb-77e4-40b4-97ac-258d876ab023 None admin] RBAC: Authorizing `identity:get_user(user_id=1c21d5d72b364361a4e68f22dd696429)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:42.690835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e66eb-77e4-40b4-97ac-258d876ab023 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:42.692997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 161/333] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:42 2022] GET /identity/v3/users/1c21d5d72b364361a4e68f22dd696429 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:42.698686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f7950-a75d-4e0b-910c-566a0bcc0f5b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:42.801451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f7950-a75d-4e0b-910c-566a0bcc0f5b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:42.838226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94e7d6-c790-430e-8eed-acdc1082b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:42.839415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94e7d6-c790-430e-8eed-acdc1082b16f None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:42.839698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94e7d6-c790-430e-8eed-acdc1082b16f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:42.839973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94e7d6-c790-430e-8eed-acdc1082b16f None admin] PATH_INFO: `/v3/system/users/1c21d5d72b364361a4e68f22dd696429/roles/c6d29efc5be34cc097e61371c95cd446` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:42.842237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94e7d6-c790-430e-8eed-acdc1082b16f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1c21d5d72b364361a4e68f22dd696429, role_id=c6d29efc5be34cc097e61371c95cd446)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:42.844228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94e7d6-c790-430e-8eed-acdc1082b16f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:42.860396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 173/334] 10.222.0.50 () {58 vars in 1334 bytes} [Fri Apr 29 18:29:42 2022] PUT /identity/v3/system/users/1c21d5d72b364361a4e68f22dd696429/roles/c6d29efc5be34cc097e61371c95cd446 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:44.967162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9103345-11b4-4ec7-b4bf-49d69ae12d3f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:44.968607 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9103345-11b4-4ec7-b4bf-49d69ae12d3f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:44.969099 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9103345-11b4-4ec7-b4bf-49d69ae12d3f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:44.970958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 162/335] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:44.976234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8d3ab-3782-4d27-b4d4-f45f998de80c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:44.977116 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8d3ab-3782-4d27-b4d4-f45f998de80c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:44.977593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8d3ab-3782-4d27-b4d4-f45f998de80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.001540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-67a8d3ab-3782-4d27-b4d4-f45f998de80c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:45.003033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-67a8d3ab-3782-4d27-b4d4-f45f998de80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:45.077322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 174/336] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:45.112768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9103345-11b4-4ec7-b4bf-49d69ae12d3f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:45.180827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa4b42-37af-4334-a4a5-008c8349cbd7 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.181525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa4b42-37af-4334-a4a5-008c8349cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.182031 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa4b42-37af-4334-a4a5-008c8349cbd7 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.183740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 175/337] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:45.198574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f492b0-fa7d-43a2-a607-8096ddcf9c0f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.198873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f492b0-fa7d-43a2-a607-8096ddcf9c0f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.199267 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f492b0-fa7d-43a2-a607-8096ddcf9c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.237629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-99f492b0-fa7d-43a2-a607-8096ddcf9c0f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:45.238817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-99f492b0-fa7d-43a2-a607-8096ddcf9c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:45.240061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9103345-11b4-4ec7-b4bf-49d69ae12d3f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:45.277391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c516f-7329-4042-99e2-b3f1a13d0028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:45.278535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c516f-7329-4042-99e2-b3f1a13d0028 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.278995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c516f-7329-4042-99e2-b3f1a13d0028 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.279454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c516f-7329-4042-99e2-b3f1a13d0028 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.293165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c516f-7329-4042-99e2-b3f1a13d0028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-130c516f-7329-4042-99e2-b3f1a13d0028 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:45.301663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:45.302662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:45.303478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:45.306382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 163/338] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:45.313059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c516f-7329-4042-99e2-b3f1a13d0028 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:45.339601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 176/339] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:45.379112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f492b0-fa7d-43a2-a607-8096ddcf9c0f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:45.415644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c516f-7329-4042-99e2-b3f1a13d0028 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:45.451008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18c845-0a6b-4386-b5ac-57240c199b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:45.452086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18c845-0a6b-4386-b5ac-57240c199b3f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.452593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18c845-0a6b-4386-b5ac-57240c199b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.453074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18c845-0a6b-4386-b5ac-57240c199b3f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.454023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18c845-0a6b-4386-b5ac-57240c199b3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:45.454527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18c845-0a6b-4386-b5ac-57240c199b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:45.456101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18c845-0a6b-4386-b5ac-57240c199b3f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:45.465353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 164/340] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:45.466941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612326a2-e94f-476e-ab77-f4a88fd47b7c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.467451 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612326a2-e94f-476e-ab77-f4a88fd47b7c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.467914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612326a2-e94f-476e-ab77-f4a88fd47b7c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.474021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 165/341] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:45.475603 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612326a2-e94f-476e-ab77-f4a88fd47b7c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:45.487738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f492b0-fa7d-43a2-a607-8096ddcf9c0f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:45.528407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29845bb-1e31-48db-b548-ce2708cfb459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:45.529236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29845bb-1e31-48db-b548-ce2708cfb459 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.529509 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29845bb-1e31-48db-b548-ce2708cfb459 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.529774 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29845bb-1e31-48db-b548-ce2708cfb459 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.546208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29845bb-1e31-48db-b548-ce2708cfb459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d29845bb-1e31-48db-b548-ce2708cfb459 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:45.558018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:45.558933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:45.559730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 177/342] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:45.560419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1779a-2de4-47b2-86c3-b15c8540af64 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.560991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1779a-2de4-47b2-86c3-b15c8540af64 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.560991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1779a-2de4-47b2-86c3-b15c8540af64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.590340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c4d1779a-2de4-47b2-86c3-b15c8540af64 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:45.591564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c4d1779a-2de4-47b2-86c3-b15c8540af64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:45.600029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612326a2-e94f-476e-ab77-f4a88fd47b7c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:45.637709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c68b1-e2cf-4c14-a24e-1d31bf2b4bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:45.638766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c68b1-e2cf-4c14-a24e-1d31bf2b4bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.639260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c68b1-e2cf-4c14-a24e-1d31bf2b4bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.639734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c68b1-e2cf-4c14-a24e-1d31bf2b4bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.690937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 178/343] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:45.691507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1779a-2de4-47b2-86c3-b15c8540af64 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:45.710711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c68b1-e2cf-4c14-a24e-1d31bf2b4bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:45.712572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c68b1-e2cf-4c14-a24e-1d31bf2b4bf2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:45.788546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1779a-2de4-47b2-86c3-b15c8540af64 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:45.814735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 166/344] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Apr 29 18:29:45.815939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c68b1-e2cf-4c14-a24e-1d31bf2b4bf2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:45.831043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387a4ab-4826-4f00-abae-7a25eb06a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:45.831730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387a4ab-4826-4f00-abae-7a25eb06a027 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.832007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387a4ab-4826-4f00-abae-7a25eb06a027 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.832277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387a4ab-4826-4f00-abae-7a25eb06a027 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.833046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387a4ab-4826-4f00-abae-7a25eb06a027 None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:45.833344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387a4ab-4826-4f00-abae-7a25eb06a027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:45.834738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387a4ab-4826-4f00-abae-7a25eb06a027 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:45.852635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 179/345] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:45.853526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387a4ab-4826-4f00-abae-7a25eb06a027 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:45.926560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c68b1-e2cf-4c14-a24e-1d31bf2b4bf2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:45.962291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442a07-53ad-4c0f-80ab-cb4eae7e3dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:45.963515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442a07-53ad-4c0f-80ab-cb4eae7e3dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.963629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387a4ab-4826-4f00-abae-7a25eb06a027 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:45.964098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442a07-53ad-4c0f-80ab-cb4eae7e3dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.964594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442a07-53ad-4c0f-80ab-cb4eae7e3dd6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.975465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71442a07-53ad-4c0f-80ab-cb4eae7e3dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-71442a07-53ad-4c0f-80ab-cb4eae7e3dd6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:45.984074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:45.985124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Apr 29 18:29:45.985929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:45.988642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 167/346] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:45.990183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5814f1-2265-4590-8efd-c8520bd59378 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:45.990656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5814f1-2265-4590-8efd-c8520bd59378 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:45.991130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5814f1-2265-4590-8efd-c8520bd59378 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:45.992585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 168/347] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:45.993457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5814f1-2265-4590-8efd-c8520bd59378 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:46.007346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f97247-6fa9-402e-b39a-4fdc26479fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:46.008075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f97247-6fa9-402e-b39a-4fdc26479fdd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:46.008350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f97247-6fa9-402e-b39a-4fdc26479fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:46.008658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f97247-6fa9-402e-b39a-4fdc26479fdd None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:46.009984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f97247-6fa9-402e-b39a-4fdc26479fdd None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:46.011582 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f97247-6fa9-402e-b39a-4fdc26479fdd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:46.013521 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 180/348] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:45 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:46.014585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de300ef9-27eb-40ad-b38f-abbe5b6c39e6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:46.014869 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de300ef9-27eb-40ad-b38f-abbe5b6c39e6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:46.015136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de300ef9-27eb-40ad-b38f-abbe5b6c39e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:46.037813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-de300ef9-27eb-40ad-b38f-abbe5b6c39e6 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:46.039444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-de300ef9-27eb-40ad-b38f-abbe5b6c39e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:46.112816 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 181/349] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:46.114216 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de300ef9-27eb-40ad-b38f-abbe5b6c39e6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:46.121448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5814f1-2265-4590-8efd-c8520bd59378 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:46.167288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d2617-f36d-4e96-8c6f-40446b8bc370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:46.167943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d2617-f36d-4e96-8c6f-40446b8bc370 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:46.168390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d2617-f36d-4e96-8c6f-40446b8bc370 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:46.168689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d2617-f36d-4e96-8c6f-40446b8bc370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:46.244941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d2617-f36d-4e96-8c6f-40446b8bc370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:46.246412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d2617-f36d-4e96-8c6f-40446b8bc370 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:46.253231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de300ef9-27eb-40ad-b38f-abbe5b6c39e6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:46.294594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6b3fd-2070-4943-8d14-7ac8889aafd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:46.295662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6b3fd-2070-4943-8d14-7ac8889aafd0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:46.296142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6b3fd-2070-4943-8d14-7ac8889aafd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:46.296647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6b3fd-2070-4943-8d14-7ac8889aafd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:46.297583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6b3fd-2070-4943-8d14-7ac8889aafd0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:46.298090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6b3fd-2070-4943-8d14-7ac8889aafd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:46.299715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6b3fd-2070-4943-8d14-7ac8889aafd0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:46.308884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 182/350] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:46.310206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6b3fd-2070-4943-8d14-7ac8889aafd0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:46.356706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 169/351] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:46.357634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d2617-f36d-4e96-8c6f-40446b8bc370 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:46.413125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6b3fd-2070-4943-8d14-7ac8889aafd0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:46.455842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b70b5-9cc7-4051-aa3a-117239b15d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:46.456802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b70b5-9cc7-4051-aa3a-117239b15d14 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:46.457078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b70b5-9cc7-4051-aa3a-117239b15d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:46.457344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b70b5-9cc7-4051-aa3a-117239b15d14 None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:46.460147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d2617-f36d-4e96-8c6f-40446b8bc370 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:46.468434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b70b5-9cc7-4051-aa3a-117239b15d14 None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:46.469923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b70b5-9cc7-4051-aa3a-117239b15d14 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:46.472046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 183/352] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:46 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:46.472839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b70b5-9cc7-4051-aa3a-117239b15d14 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:46.498174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b124fe3-9eba-44bc-a48e-7182f68e0cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:46.499361 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b124fe3-9eba-44bc-a48e-7182f68e0cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:46.499851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b124fe3-9eba-44bc-a48e-7182f68e0cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:46.500311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b124fe3-9eba-44bc-a48e-7182f68e0cd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:46.513204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b124fe3-9eba-44bc-a48e-7182f68e0cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5b124fe3-9eba-44bc-a48e-7182f68e0cd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:46.524764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:46.525764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:46.526552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:46.529844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 170/353] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:46.530903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b124fe3-9eba-44bc-a48e-7182f68e0cd2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:46.609179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b70b5-9cc7-4051-aa3a-117239b15d14 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:46.653260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aa8a2-7265-4053-ae9b-1ffd3be7d75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:46.654147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aa8a2-7265-4053-ae9b-1ffd3be7d75d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:46.654635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aa8a2-7265-4053-ae9b-1ffd3be7d75d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:46.655090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aa8a2-7265-4053-ae9b-1ffd3be7d75d None admin] PATH_INFO: `/v3/users/1c21d5d72b364361a4e68f22dd696429` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:46.656255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b124fe3-9eba-44bc-a48e-7182f68e0cd2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:46.656797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598aa8a2-7265-4053-ae9b-1ffd3be7d75d None admin] RBAC: Authorizing `identity:get_user(user_id=1c21d5d72b364361a4e68f22dd696429)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:46.658443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598aa8a2-7265-4053-ae9b-1ffd3be7d75d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:46.660324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 184/354] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:29:46 2022] GET /identity/v3/users/1c21d5d72b364361a4e68f22dd696429 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:46.661418 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aa8a2-7265-4053-ae9b-1ffd3be7d75d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:46.694017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365541e-a31b-4640-800d-6ba660b7eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:46.695412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365541e-a31b-4640-800d-6ba660b7eed1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:46.695876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365541e-a31b-4640-800d-6ba660b7eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:46.696352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365541e-a31b-4640-800d-6ba660b7eed1 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:46.697949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365541e-a31b-4640-800d-6ba660b7eed1 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:46.699700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365541e-a31b-4640-800d-6ba660b7eed1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:46.702835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 171/355] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:46 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:46.703833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365541e-a31b-4640-800d-6ba660b7eed1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:46.787454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aa8a2-7265-4053-ae9b-1ffd3be7d75d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:46.827552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365541e-a31b-4640-800d-6ba660b7eed1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:46.829133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fa186-26b3-4b1a-b789-80e8737ecc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:46.830019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fa186-26b3-4b1a-b789-80e8737ecc7e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:46.830301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fa186-26b3-4b1a-b789-80e8737ecc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:46.830591 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fa186-26b3-4b1a-b789-80e8737ecc7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:46.831433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8fa186-26b3-4b1a-b789-80e8737ecc7e None admin] RBAC: Adding query filter params (group.id=4286f4b1241146f0a49d724fe365a5bf, role.id=3385b41b990e4c6b88d77260a7822c7e, scope.project.id=7a48da3b90914e098e862c1aadf1620b) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:46.831737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8fa186-26b3-4b1a-b789-80e8737ecc7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:46.835246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8fa186-26b3-4b1a-b789-80e8737ecc7e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:46.841754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 185/356] 10.222.0.50 () {56 vars in 1376 bytes} [Fri Apr 29 18:29:46 2022] GET /identity/v3/role_assignments?group.id=4286f4b1241146f0a49d724fe365a5bf&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:46.842375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fa186-26b3-4b1a-b789-80e8737ecc7e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:46.875855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190284bb-07c1-402e-94f1-6d698799d82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:46.876878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190284bb-07c1-402e-94f1-6d698799d82b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:46.877359 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190284bb-07c1-402e-94f1-6d698799d82b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:46.877806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190284bb-07c1-402e-94f1-6d698799d82b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:46.878749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190284bb-07c1-402e-94f1-6d698799d82b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:46.879238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190284bb-07c1-402e-94f1-6d698799d82b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:46.881261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190284bb-07c1-402e-94f1-6d698799d82b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:46.891595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 172/357] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:46.892935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190284bb-07c1-402e-94f1-6d698799d82b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:46.974960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fa186-26b3-4b1a-b789-80e8737ecc7e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:47.014695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190284bb-07c1-402e-94f1-6d698799d82b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:47.015821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f25bf8-f1fa-4e20-9df4-d4e40b55160d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:47.017042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25bf8-f1fa-4e20-9df4-d4e40b55160d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.017463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25bf8-f1fa-4e20-9df4-d4e40b55160d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.017885 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25bf8-f1fa-4e20-9df4-d4e40b55160d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.018872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f25bf8-f1fa-4e20-9df4-d4e40b55160d None admin] RBAC: Adding query filter params (role.id=c6d29efc5be34cc097e61371c95cd446, user.id=1c21d5d72b364361a4e68f22dd696429, scope.system=all) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:47.019403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f25bf8-f1fa-4e20-9df4-d4e40b55160d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:47.021162 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f25bf8-f1fa-4e20-9df4-d4e40b55160d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:47.034539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 186/358] 10.222.0.50 () {56 vars in 1308 bytes} [Fri Apr 29 18:29:46 2022] GET /identity/v3/role_assignments?user.id=1c21d5d72b364361a4e68f22dd696429&scope.system=all&role.id=c6d29efc5be34cc097e61371c95cd446 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:47.036269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2317db-e947-4164-a6f9-d19a34854172 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.036806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2317db-e947-4164-a6f9-d19a34854172 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.037299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2317db-e947-4164-a6f9-d19a34854172 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.039398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 187/359] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:47.044768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2317db-e947-4164-a6f9-d19a34854172 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:47.054473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ec909-4efd-421f-b66c-2e86670c842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:47.055429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ec909-4efd-421f-b66c-2e86670c842d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.055756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ec909-4efd-421f-b66c-2e86670c842d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.056120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ec909-4efd-421f-b66c-2e86670c842d None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.057609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ec909-4efd-421f-b66c-2e86670c842d None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:47.059435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ec909-4efd-421f-b66c-2e86670c842d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:47.061549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 173/360] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:46 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:47.063037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558ad6e-dac6-481a-8867-7b4e11edd7d1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.063566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558ad6e-dac6-481a-8867-7b4e11edd7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.064056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558ad6e-dac6-481a-8867-7b4e11edd7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.083864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3558ad6e-dac6-481a-8867-7b4e11edd7d1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:47.085440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3558ad6e-dac6-481a-8867-7b4e11edd7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:47.156932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 174/361] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:47.158131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558ad6e-dac6-481a-8867-7b4e11edd7d1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:47.165815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2317db-e947-4164-a6f9-d19a34854172 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:47.217964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab529d17-5f56-4ea4-987e-e3b2a0e7df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:47.218996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab529d17-5f56-4ea4-987e-e3b2a0e7df25 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.219274 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab529d17-5f56-4ea4-987e-e3b2a0e7df25 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.219843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab529d17-5f56-4ea4-987e-e3b2a0e7df25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.264752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558ad6e-dac6-481a-8867-7b4e11edd7d1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:47.289504 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab529d17-5f56-4ea4-987e-e3b2a0e7df25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:47.291021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab529d17-5f56-4ea4-987e-e3b2a0e7df25 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:47.301178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d542ce-cb3a-494b-8305-7149a6d89c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:47.302379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d542ce-cb3a-494b-8305-7149a6d89c11 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.302915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d542ce-cb3a-494b-8305-7149a6d89c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.303369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d542ce-cb3a-494b-8305-7149a6d89c11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.304402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d542ce-cb3a-494b-8305-7149a6d89c11 None admin] RBAC: Adding query filter params (group.id=4286f4b1241146f0a49d724fe365a5bf, role.id=3385b41b990e4c6b88d77260a7822c7e, scope.project.id=24cf0228dd084700bf349d5310285eb0) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:47.304942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d542ce-cb3a-494b-8305-7149a6d89c11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:47.306792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d542ce-cb3a-494b-8305-7149a6d89c11 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:47.313312 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 175/362] 10.222.0.50 () {56 vars in 1376 bytes} [Fri Apr 29 18:29:47 2022] GET /identity/v3/role_assignments?group.id=4286f4b1241146f0a49d724fe365a5bf&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 260 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:47.314392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d542ce-cb3a-494b-8305-7149a6d89c11 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:47.425678 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 188/363] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:47.427431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d10bf-ad8d-40d6-8887-65520a7b99a6 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.427929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d10bf-ad8d-40d6-8887-65520a7b99a6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.428395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d10bf-ad8d-40d6-8887-65520a7b99a6 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.429949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 189/364] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:47.431155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d542ce-cb3a-494b-8305-7149a6d89c11 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:47.436938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d10bf-ad8d-40d6-8887-65520a7b99a6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:47.475686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131968ee-4612-404a-bce3-747ccc0e367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:47.477064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131968ee-4612-404a-bce3-747ccc0e367b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.477553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131968ee-4612-404a-bce3-747ccc0e367b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.478052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131968ee-4612-404a-bce3-747ccc0e367b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.494327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131968ee-4612-404a-bce3-747ccc0e367b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-131968ee-4612-404a-bce3-747ccc0e367b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:47.505315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:47.506279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:47.507108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:47.510784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 176/365] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:47.512539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a66522-eefb-42e1-8fb2-99f1ef9c4821 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.513044 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a66522-eefb-42e1-8fb2-99f1ef9c4821 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.513503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a66522-eefb-42e1-8fb2-99f1ef9c4821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.534840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-91a66522-eefb-42e1-8fb2-99f1ef9c4821 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:47.537024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-91a66522-eefb-42e1-8fb2-99f1ef9c4821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:47.561017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d10bf-ad8d-40d6-8887-65520a7b99a6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:47.604479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bfc05-4303-4de9-b3a3-7d892dff6b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:47.604479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bfc05-4303-4de9-b3a3-7d892dff6b11 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.604479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bfc05-4303-4de9-b3a3-7d892dff6b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.604832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bfc05-4303-4de9-b3a3-7d892dff6b11 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.605892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bfc05-4303-4de9-b3a3-7d892dff6b11 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:47.607359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bfc05-4303-4de9-b3a3-7d892dff6b11 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:47.612171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 190/366] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:47 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:47.612171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bfc05-4303-4de9-b3a3-7d892dff6b11 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:47.662476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 177/367] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:47.663684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a66522-eefb-42e1-8fb2-99f1ef9c4821 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:47.753525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bfc05-4303-4de9-b3a3-7d892dff6b11 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:47.781888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a66522-eefb-42e1-8fb2-99f1ef9c4821 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:47.795148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9318018-07b1-49ca-a8af-b9092fb6a11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:47.795843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9318018-07b1-49ca-a8af-b9092fb6a11d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.796124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9318018-07b1-49ca-a8af-b9092fb6a11d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.796397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9318018-07b1-49ca-a8af-b9092fb6a11d None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.797390 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9318018-07b1-49ca-a8af-b9092fb6a11d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:47.797625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9318018-07b1-49ca-a8af-b9092fb6a11d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:47.799113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9318018-07b1-49ca-a8af-b9092fb6a11d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:47.811202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 191/368] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:47.811848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9318018-07b1-49ca-a8af-b9092fb6a11d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:47.832169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9364ecd-38db-43b9-bf8f-48016eb8f6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:47.833641 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9364ecd-38db-43b9-bf8f-48016eb8f6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.834190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9364ecd-38db-43b9-bf8f-48016eb8f6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.834624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9364ecd-38db-43b9-bf8f-48016eb8f6b5 None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.836413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9364ecd-38db-43b9-bf8f-48016eb8f6b5 None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:47.838313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9364ecd-38db-43b9-bf8f-48016eb8f6b5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:47.841112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 178/369] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:47 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:47.842350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9364ecd-38db-43b9-bf8f-48016eb8f6b5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:47.929512 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9318018-07b1-49ca-a8af-b9092fb6a11d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:47.954321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9364ecd-38db-43b9-bf8f-48016eb8f6b5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:47.972676 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840815f-4be5-44cb-81f0-03f967e2ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:47.973859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840815f-4be5-44cb-81f0-03f967e2ad98 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.974347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840815f-4be5-44cb-81f0-03f967e2ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.974779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840815f-4be5-44cb-81f0-03f967e2ad98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.988749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840815f-4be5-44cb-81f0-03f967e2ad98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:47.990499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81abed-8ca2-423f-9ec9-24d12f5824a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:47.991208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81abed-8ca2-423f-9ec9-24d12f5824a9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:47.991556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81abed-8ca2-423f-9ec9-24d12f5824a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:47.991943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81abed-8ca2-423f-9ec9-24d12f5824a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-a840815f-4be5-44cb-81f0-03f967e2ad98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:47.999288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:48.000176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:48.000987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:48.003345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 192/370] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:29:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:48.004355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840815f-4be5-44cb-81f0-03f967e2ad98 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:48.080194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81abed-8ca2-423f-9ec9-24d12f5824a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:48.082251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81abed-8ca2-423f-9ec9-24d12f5824a9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:48.120089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840815f-4be5-44cb-81f0-03f967e2ad98 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:48.156122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d851c4-1f46-4018-961b-7b9aeaf9ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:48.156974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d851c4-1f46-4018-961b-7b9aeaf9ab66 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:48.157353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d851c4-1f46-4018-961b-7b9aeaf9ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:48.157723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d851c4-1f46-4018-961b-7b9aeaf9ab66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:48.158939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d851c4-1f46-4018-961b-7b9aeaf9ab66 None admin] RBAC: Adding query filter params (group.id=4286f4b1241146f0a49d724fe365a5bf, role.id=ec08a86158a44ee2b82951b44b228df2, scope.project.id=7a48da3b90914e098e862c1aadf1620b) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:48.158939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d851c4-1f46-4018-961b-7b9aeaf9ab66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:48.161343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d851c4-1f46-4018-961b-7b9aeaf9ab66 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:48.167921 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 193/371] 10.222.0.50 () {56 vars in 1376 bytes} [Fri Apr 29 18:29:48 2022] GET /identity/v3/role_assignments?group.id=4286f4b1241146f0a49d724fe365a5bf&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=ec08a86158a44ee2b82951b44b228df2 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:48.169002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d851c4-1f46-4018-961b-7b9aeaf9ab66 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:48.189176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 179/372] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:48.195145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81abed-8ca2-423f-9ec9-24d12f5824a9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:48.351799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d851c4-1f46-4018-961b-7b9aeaf9ab66 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:48.378340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81abed-8ca2-423f-9ec9-24d12f5824a9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:48.424635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aeb41-a95b-424b-8b9b-189fd20dd523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:48.425684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aeb41-a95b-424b-8b9b-189fd20dd523 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:48.426172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aeb41-a95b-424b-8b9b-189fd20dd523 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:48.426645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aeb41-a95b-424b-8b9b-189fd20dd523 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:48.427677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aeb41-a95b-424b-8b9b-189fd20dd523 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:48.428170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aeb41-a95b-424b-8b9b-189fd20dd523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:48.429795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aeb41-a95b-424b-8b9b-189fd20dd523 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:48.439826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f894b6-8efe-411f-9bef-b26327326766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:48.440733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f894b6-8efe-411f-9bef-b26327326766 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:48.441010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f894b6-8efe-411f-9bef-b26327326766 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:48.441279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f894b6-8efe-411f-9bef-b26327326766 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:48.442592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f894b6-8efe-411f-9bef-b26327326766 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:48.444070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f894b6-8efe-411f-9bef-b26327326766 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:48.445868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 180/373] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:48 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:48.449123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 194/374] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:48.452847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f894b6-8efe-411f-9bef-b26327326766 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:48.454574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aeb41-a95b-424b-8b9b-189fd20dd523 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:48.565753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f894b6-8efe-411f-9bef-b26327326766 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:48.613906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248142f5-0e91-4bc7-bb5d-e414c0a34911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:48.615133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248142f5-0e91-4bc7-bb5d-e414c0a34911 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:48.615607 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248142f5-0e91-4bc7-bb5d-e414c0a34911 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:48.616067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248142f5-0e91-4bc7-bb5d-e414c0a34911 None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:48.617485 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248142f5-0e91-4bc7-bb5d-e414c0a34911 None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:48.619220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248142f5-0e91-4bc7-bb5d-e414c0a34911 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:48.621450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 181/375] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:48 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:48.628143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248142f5-0e91-4bc7-bb5d-e414c0a34911 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:48.634340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aeb41-a95b-424b-8b9b-189fd20dd523 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:48.671422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b6c99-95c3-4c46-b086-6d75edd8f2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:48.672096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b6c99-95c3-4c46-b086-6d75edd8f2ec None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:48.672425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b6c99-95c3-4c46-b086-6d75edd8f2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:48.672773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b6c99-95c3-4c46-b086-6d75edd8f2ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:48.734164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248142f5-0e91-4bc7-bb5d-e414c0a34911 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:48.748688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b6c99-95c3-4c46-b086-6d75edd8f2ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:48.750188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b6c99-95c3-4c46-b086-6d75edd8f2ec None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:48.775998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f89523-e002-4801-8e22-61708e7f15ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:48.777088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f89523-e002-4801-8e22-61708e7f15ef None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:48.777589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f89523-e002-4801-8e22-61708e7f15ef None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:48.778052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f89523-e002-4801-8e22-61708e7f15ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:48.779081 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f89523-e002-4801-8e22-61708e7f15ef None admin] RBAC: Adding query filter params (group.id=4286f4b1241146f0a49d724fe365a5bf, role.id=ec08a86158a44ee2b82951b44b228df2, scope.project.id=24cf0228dd084700bf349d5310285eb0) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:48.779595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f89523-e002-4801-8e22-61708e7f15ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:48.781300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f89523-e002-4801-8e22-61708e7f15ef None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:48.792735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 182/376] 10.222.0.50 () {56 vars in 1376 bytes} [Fri Apr 29 18:29:48 2022] GET /identity/v3/role_assignments?group.id=4286f4b1241146f0a49d724fe365a5bf&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=ec08a86158a44ee2b82951b44b228df2 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:48.866091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 195/377] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Apr 29 18:29:48.867496 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f89523-e002-4801-8e22-61708e7f15ef None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:49.020649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f89523-e002-4801-8e22-61708e7f15ef None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:49.059809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fa196-ca31-40a5-bf98-a953e1e91883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:49.061114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fa196-ca31-40a5-bf98-a953e1e91883 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:49.061608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fa196-ca31-40a5-bf98-a953e1e91883 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:49.062071 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fa196-ca31-40a5-bf98-a953e1e91883 None admin] PATH_INFO: `/v3/projects/28300f353c4c4c27a229238b2a238c8e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:49.064633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fa196-ca31-40a5-bf98-a953e1e91883 None admin] RBAC: Authorizing `identity:get_project(project_id=28300f353c4c4c27a229238b2a238c8e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:49.066301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fa196-ca31-40a5-bf98-a953e1e91883 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:49.069685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 183/378] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:48 2022] GET /identity/v3/projects/28300f353c4c4c27a229238b2a238c8e => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:49.075614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b6c99-95c3-4c46-b086-6d75edd8f2ec None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:49.186618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b6c99-95c3-4c46-b086-6d75edd8f2ec None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:49.224022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16d579-949d-491e-b2d2-a399bf59d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:49.225333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16d579-949d-491e-b2d2-a399bf59d26c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:49.225838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16d579-949d-491e-b2d2-a399bf59d26c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:49.226303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16d579-949d-491e-b2d2-a399bf59d26c None admin] PATH_INFO: `/v3/groups/867b42672a0f4f0c9268fbbd67a6368b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:49.231832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16d579-949d-491e-b2d2-a399bf59d26c None admin] RBAC: Authorizing `identity:get_group(group_id=867b42672a0f4f0c9268fbbd67a6368b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:49.233653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16d579-949d-491e-b2d2-a399bf59d26c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:49.235584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 196/379] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:49 2022] GET /identity/v3/groups/867b42672a0f4f0c9268fbbd67a6368b => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:49.241878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fa196-ca31-40a5-bf98-a953e1e91883 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:49.356201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fa196-ca31-40a5-bf98-a953e1e91883 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:49.400868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd23205-d27c-4655-9e58-ffbe530ec064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:49.401932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd23205-d27c-4655-9e58-ffbe530ec064 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:49.402432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd23205-d27c-4655-9e58-ffbe530ec064 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:49.402901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd23205-d27c-4655-9e58-ffbe530ec064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:49.403942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd23205-d27c-4655-9e58-ffbe530ec064 None admin] RBAC: Adding query filter params (group.id=867b42672a0f4f0c9268fbbd67a6368b, role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=28300f353c4c4c27a229238b2a238c8e) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:49.404447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd23205-d27c-4655-9e58-ffbe530ec064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:49.406098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd23205-d27c-4655-9e58-ffbe530ec064 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:49.414840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 184/380] 10.222.0.50 () {56 vars in 1376 bytes} [Fri Apr 29 18:29:49 2022] GET /identity/v3/role_assignments?group.id=867b42672a0f4f0c9268fbbd67a6368b&scope.project.id=28300f353c4c4c27a229238b2a238c8e&role.id=e55ef11844a742f5889243f189bd59bc => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:49.746312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e615e8-2c65-4129-bd8b-732e52bc2714 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:49.747054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e615e8-2c65-4129-bd8b-732e52bc2714 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:49.747537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e615e8-2c65-4129-bd8b-732e52bc2714 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:49.749278 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 197/381] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:49.755373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead4a25-9f81-4839-b48d-588c894d0d9d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:49.756003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead4a25-9f81-4839-b48d-588c894d0d9d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:49.756544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead4a25-9f81-4839-b48d-588c894d0d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:49.780388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bead4a25-9f81-4839-b48d-588c894d0d9d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:49.781932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bead4a25-9f81-4839-b48d-588c894d0d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:49.857756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 185/382] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:49.880732 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e615e8-2c65-4129-bd8b-732e52bc2714 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:49.997013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e615e8-2c65-4129-bd8b-732e52bc2714 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:50.062147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524536d-ba82-4efa-94c7-b5d11b0d5f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:50.062985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524536d-ba82-4efa-94c7-b5d11b0d5f0e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:50.063254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524536d-ba82-4efa-94c7-b5d11b0d5f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:50.063518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524536d-ba82-4efa-94c7-b5d11b0d5f0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:50.087979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524536d-ba82-4efa-94c7-b5d11b0d5f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0524536d-ba82-4efa-94c7-b5d11b0d5f0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:50.096063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:50.097010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:50.098220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 198/383] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:50.103890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead4a25-9f81-4839-b48d-588c894d0d9d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:50.276896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead4a25-9f81-4839-b48d-588c894d0d9d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:50.322868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f24f56-9d6b-424c-b61c-2916ee3ec8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:50.323960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f24f56-9d6b-424c-b61c-2916ee3ec8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:50.324476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f24f56-9d6b-424c-b61c-2916ee3ec8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:50.324974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f24f56-9d6b-424c-b61c-2916ee3ec8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:50.325920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f24f56-9d6b-424c-b61c-2916ee3ec8d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:50.326415 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f24f56-9d6b-424c-b61c-2916ee3ec8d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:50.327976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f24f56-9d6b-424c-b61c-2916ee3ec8d2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:50.340422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 186/384] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:50.346140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524536d-ba82-4efa-94c7-b5d11b0d5f0e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:50.478974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020af0c-0887-413c-8550-c257f149a7f1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:50.479686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020af0c-0887-413c-8550-c257f149a7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:50.480214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020af0c-0887-413c-8550-c257f149a7f1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:50.486734 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524536d-ba82-4efa-94c7-b5d11b0d5f0e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:50.487020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 187/385] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:29:50.492688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d29dc-6c58-4677-8809-1d7f12539502 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:50.493223 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d29dc-6c58-4677-8809-1d7f12539502 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:50.493808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d29dc-6c58-4677-8809-1d7f12539502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:50.525762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0f1d29dc-6c58-4677-8809-1d7f12539502 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:50.527354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0f1d29dc-6c58-4677-8809-1d7f12539502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:50.529781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b013d6-223c-4d15-9495-7c0d3dd3b1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:50.530948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b013d6-223c-4d15-9495-7c0d3dd3b1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:50.531229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b013d6-223c-4d15-9495-7c0d3dd3b1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:50.532376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b013d6-223c-4d15-9495-7c0d3dd3b1c7 None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:50.538141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b013d6-223c-4d15-9495-7c0d3dd3b1c7 None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:50.539082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b013d6-223c-4d15-9495-7c0d3dd3b1c7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:50.545887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 199/386] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:50 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:50.548726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b013d6-223c-4d15-9495-7c0d3dd3b1c7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:50.647191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 188/387] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:50.658474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b013d6-223c-4d15-9495-7c0d3dd3b1c7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:50.687712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d29dc-6c58-4677-8809-1d7f12539502 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:50.695542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b82b2-3268-49d4-8f60-0ae0447cb2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:50.696445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b82b2-3268-49d4-8f60-0ae0447cb2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:50.696967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b82b2-3268-49d4-8f60-0ae0447cb2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:50.697361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b82b2-3268-49d4-8f60-0ae0447cb2e4 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:50.701274 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b82b2-3268-49d4-8f60-0ae0447cb2e4 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:50.702743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b82b2-3268-49d4-8f60-0ae0447cb2e4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:50.704515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 200/388] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:50 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:50.710511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b82b2-3268-49d4-8f60-0ae0447cb2e4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:50.802987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d29dc-6c58-4677-8809-1d7f12539502 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:50.815784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b82b2-3268-49d4-8f60-0ae0447cb2e4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:50.846644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a32e6-6f94-4d55-94b6-3766563f152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:50.847754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a32e6-6f94-4d55-94b6-3766563f152a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:50.848193 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a32e6-6f94-4d55-94b6-3766563f152a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:50.848637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a32e6-6f94-4d55-94b6-3766563f152a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:50.857567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b46c3-8222-4bb4-9f52-b82bd94e909f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:50.860209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b46c3-8222-4bb4-9f52-b82bd94e909f None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:50.860709 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b46c3-8222-4bb4-9f52-b82bd94e909f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:50.861129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b46c3-8222-4bb4-9f52-b82bd94e909f None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b/groups/4286f4b1241146f0a49d724fe365a5bf/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:50.865185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b46c3-8222-4bb4-9f52-b82bd94e909f None admin] RBAC: Authorizing `identity:create_grant(project_id=7a48da3b90914e098e862c1aadf1620b, group_id=4286f4b1241146f0a49d724fe365a5bf, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:50.868156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b46c3-8222-4bb4-9f52-b82bd94e909f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:50.873951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a32e6-6f94-4d55-94b6-3766563f152a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f34a32e6-6f94-4d55-94b6-3766563f152a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:50.882442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:50.883365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:50.884233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:50.887484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 189/389] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:50.893808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a32e6-6f94-4d55-94b6-3766563f152a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:50.894391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 201/390] 10.222.0.50 () {58 vars in 1442 bytes} [Fri Apr 29 18:29:50 2022] PUT /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b/groups/4286f4b1241146f0a49d724fe365a5bf/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:51.006988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a32e6-6f94-4d55-94b6-3766563f152a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:51.045840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845ac39-c021-443e-b966-ee84d089767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:51.046867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845ac39-c021-443e-b966-ee84d089767f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:51.047320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845ac39-c021-443e-b966-ee84d089767f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:51.047828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845ac39-c021-443e-b966-ee84d089767f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:51.048836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845ac39-c021-443e-b966-ee84d089767f None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:51.049313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845ac39-c021-443e-b966-ee84d089767f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:51.050876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845ac39-c021-443e-b966-ee84d089767f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:51.062049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 190/391] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:51.072802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b46c3-8222-4bb4-9f52-b82bd94e909f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:51.236380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b46c3-8222-4bb4-9f52-b82bd94e909f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:51.279068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ccdc6-e0be-4f66-8407-8f8a9acff80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:51.279997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ccdc6-e0be-4f66-8407-8f8a9acff80b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:51.280284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ccdc6-e0be-4f66-8407-8f8a9acff80b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:51.280587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ccdc6-e0be-4f66-8407-8f8a9acff80b None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:51.281764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ccdc6-e0be-4f66-8407-8f8a9acff80b None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:51.283221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ccdc6-e0be-4f66-8407-8f8a9acff80b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:51.288600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 202/392] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:51 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:51.291286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845ac39-c021-443e-b966-ee84d089767f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:51.362428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e7ce3-54b4-42b1-91aa-6b2a9c15e220 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:51.362935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e7ce3-54b4-42b1-91aa-6b2a9c15e220 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:51.362935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e7ce3-54b4-42b1-91aa-6b2a9c15e220 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:51.364436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 203/393] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:29:51.371090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f7c8-85a9-44ad-883f-a23e6df525fc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:51.371380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f7c8-85a9-44ad-883f-a23e6df525fc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:51.371654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f7c8-85a9-44ad-883f-a23e6df525fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:51.403097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8627f7c8-85a9-44ad-883f-a23e6df525fc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:51.404298 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8627f7c8-85a9-44ad-883f-a23e6df525fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:51.413683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845ac39-c021-443e-b966-ee84d089767f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:51.464895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf4aee-3713-4b31-9fb2-36dc8f80e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:51.465780 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf4aee-3713-4b31-9fb2-36dc8f80e76a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:51.466059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf4aee-3713-4b31-9fb2-36dc8f80e76a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:51.466323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf4aee-3713-4b31-9fb2-36dc8f80e76a None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:51.469699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf4aee-3713-4b31-9fb2-36dc8f80e76a None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:51.471170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf4aee-3713-4b31-9fb2-36dc8f80e76a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:51.473139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 191/394] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:51 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:51.479482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf4aee-3713-4b31-9fb2-36dc8f80e76a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:51.521835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 204/395] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:51.560659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627f7c8-85a9-44ad-883f-a23e6df525fc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:51.663390 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627f7c8-85a9-44ad-883f-a23e6df525fc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:51.676447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf4aee-3713-4b31-9fb2-36dc8f80e76a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:51.698153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61cd91-8357-40bd-9254-c2073c812957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:51.698981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61cd91-8357-40bd-9254-c2073c812957 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:51.699262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61cd91-8357-40bd-9254-c2073c812957 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:51.699532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61cd91-8357-40bd-9254-c2073c812957 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:51.711107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61cd91-8357-40bd-9254-c2073c812957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5a61cd91-8357-40bd-9254-c2073c812957 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:51.720007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:51.720956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:51.721733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 205/396] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:51.727163 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61cd91-8357-40bd-9254-c2073c812957 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:51.735479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576ea2b-ca6a-4495-a52a-e4742b7fbecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:51.736435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ea2b-ca6a-4495-a52a-e4742b7fbecd None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:51.736743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ea2b-ca6a-4495-a52a-e4742b7fbecd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:51.737015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ea2b-ca6a-4495-a52a-e4742b7fbecd None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0/groups/4286f4b1241146f0a49d724fe365a5bf/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:51.741453 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576ea2b-ca6a-4495-a52a-e4742b7fbecd None admin] RBAC: Authorizing `identity:create_grant(project_id=24cf0228dd084700bf349d5310285eb0, group_id=4286f4b1241146f0a49d724fe365a5bf, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:51.742974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576ea2b-ca6a-4495-a52a-e4742b7fbecd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:51.776726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 192/397] 10.222.0.50 () {58 vars in 1442 bytes} [Fri Apr 29 18:29:51 2022] PUT /identity/v3/projects/24cf0228dd084700bf349d5310285eb0/groups/4286f4b1241146f0a49d724fe365a5bf/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:51.827878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61cd91-8357-40bd-9254-c2073c812957 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:51.862716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be03d61-e7b2-4120-8e52-9c3094fa8c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:51.863390 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be03d61-e7b2-4120-8e52-9c3094fa8c7a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:51.863667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be03d61-e7b2-4120-8e52-9c3094fa8c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:51.863933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be03d61-e7b2-4120-8e52-9c3094fa8c7a None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:51.864703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be03d61-e7b2-4120-8e52-9c3094fa8c7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:51.865004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be03d61-e7b2-4120-8e52-9c3094fa8c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:51.866401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be03d61-e7b2-4120-8e52-9c3094fa8c7a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:51.876096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 206/398] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:51.881799 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576ea2b-ca6a-4495-a52a-e4742b7fbecd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:51.930129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e65add-0084-44c8-8c3c-c3a4a096e901 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:51.930474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e65add-0084-44c8-8c3c-c3a4a096e901 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:51.930673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e65add-0084-44c8-8c3c-c3a4a096e901 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:51.932433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 207/399] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:51.939404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16712234-82f9-4e3a-92dc-f2c238f11606 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:51.939725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16712234-82f9-4e3a-92dc-f2c238f11606 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:51.940062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16712234-82f9-4e3a-92dc-f2c238f11606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:51.965802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-16712234-82f9-4e3a-92dc-f2c238f11606 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:51.967031 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-16712234-82f9-4e3a-92dc-f2c238f11606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:52.067479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576ea2b-ca6a-4495-a52a-e4742b7fbecd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:52.102470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 208/400] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:52.125418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3449c26-4725-49f8-bcf5-3813a6370ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:52.126350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3449c26-4725-49f8-bcf5-3813a6370ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:52.126582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3449c26-4725-49f8-bcf5-3813a6370ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:52.126857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3449c26-4725-49f8-bcf5-3813a6370ed2 None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:52.129022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3449c26-4725-49f8-bcf5-3813a6370ed2 None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:52.130486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3449c26-4725-49f8-bcf5-3813a6370ed2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:52.131939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16712234-82f9-4e3a-92dc-f2c238f11606 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:52.132651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 193/401] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:51 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:52.141596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3449c26-4725-49f8-bcf5-3813a6370ed2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:52.254082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3449c26-4725-49f8-bcf5-3813a6370ed2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:52.257225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16712234-82f9-4e3a-92dc-f2c238f11606 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:52.294095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558754b-23d9-4876-a0ae-ca395c9be3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:52.295036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558754b-23d9-4876-a0ae-ca395c9be3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:52.295317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558754b-23d9-4876-a0ae-ca395c9be3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:52.295591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558754b-23d9-4876-a0ae-ca395c9be3c4 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:52.297984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558754b-23d9-4876-a0ae-ca395c9be3c4 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:52.299487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558754b-23d9-4876-a0ae-ca395c9be3c4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:52.301765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 194/402] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:52 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:52.304125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c92ff-b00c-4a24-aa7b-f923687d9a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:52.308574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c92ff-b00c-4a24-aa7b-f923687d9a27 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:52.308574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c92ff-b00c-4a24-aa7b-f923687d9a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:52.308574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c92ff-b00c-4a24-aa7b-f923687d9a27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:52.310273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558754b-23d9-4876-a0ae-ca395c9be3c4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:52.328806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c92ff-b00c-4a24-aa7b-f923687d9a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-896c92ff-b00c-4a24-aa7b-f923687d9a27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:52.344523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:52.345495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:52.346714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 209/403] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:52.354490 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c92ff-b00c-4a24-aa7b-f923687d9a27 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:52.486756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c92ff-b00c-4a24-aa7b-f923687d9a27 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:52.525753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7e8a5-fb9a-4323-8436-e2875b5bac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:52.526739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7e8a5-fb9a-4323-8436-e2875b5bac3e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:52.527157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7e8a5-fb9a-4323-8436-e2875b5bac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:52.527543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7e8a5-fb9a-4323-8436-e2875b5bac3e None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:52.528302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7e8a5-fb9a-4323-8436-e2875b5bac3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:52.528631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7e8a5-fb9a-4323-8436-e2875b5bac3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:52.530012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7e8a5-fb9a-4323-8436-e2875b5bac3e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:52.530189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558754b-23d9-4876-a0ae-ca395c9be3c4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:52.540555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 210/404] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:52.542406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7e840-0a54-495c-b45b-8c84ca12131f None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:52.542705 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7e840-0a54-495c-b45b-8c84ca12131f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:52.543137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7e840-0a54-495c-b45b-8c84ca12131f None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:52.545053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 211/405] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:52.546412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7e840-0a54-495c-b45b-8c84ca12131f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:52.578809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9fb02-b361-46b8-a73e-41b6a4b009ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:52.579699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9fb02-b361-46b8-a73e-41b6a4b009ef None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:52.579974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9fb02-b361-46b8-a73e-41b6a4b009ef None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:52.580240 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9fb02-b361-46b8-a73e-41b6a4b009ef None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b/groups/4286f4b1241146f0a49d724fe365a5bf/roles/ec08a86158a44ee2b82951b44b228df2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:52.582511 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9fb02-b361-46b8-a73e-41b6a4b009ef None admin] RBAC: Authorizing `identity:create_grant(project_id=7a48da3b90914e098e862c1aadf1620b, group_id=4286f4b1241146f0a49d724fe365a5bf, role_id=ec08a86158a44ee2b82951b44b228df2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:52.583954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9fb02-b361-46b8-a73e-41b6a4b009ef None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:52.605293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 195/406] 10.222.0.50 () {58 vars in 1442 bytes} [Fri Apr 29 18:29:52 2022] PUT /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b/groups/4286f4b1241146f0a49d724fe365a5bf/roles/ec08a86158a44ee2b82951b44b228df2 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:52.606735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd7766-9479-45ad-8f2c-f9faa629a3ed None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:52.607018 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd7766-9479-45ad-8f2c-f9faa629a3ed None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:52.607288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd7766-9479-45ad-8f2c-f9faa629a3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:52.631687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1ddd7766-9479-45ad-8f2c-f9faa629a3ed None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:52.632968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1ddd7766-9479-45ad-8f2c-f9faa629a3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:52.666927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7e840-0a54-495c-b45b-8c84ca12131f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:52.703305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369b829-f624-420c-9fc2-553aa0104a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:52.704319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369b829-f624-420c-9fc2-553aa0104a38 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:52.704664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369b829-f624-420c-9fc2-553aa0104a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:52.704939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369b829-f624-420c-9fc2-553aa0104a38 None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:52.705221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 196/407] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:52.706144 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369b829-f624-420c-9fc2-553aa0104a38 None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:52.707717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369b829-f624-420c-9fc2-553aa0104a38 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:52.710762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 212/408] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:52 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:52.717189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd7766-9479-45ad-8f2c-f9faa629a3ed None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:52.783636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369b829-f624-420c-9fc2-553aa0104a38 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:52.830637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd7766-9479-45ad-8f2c-f9faa629a3ed None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:52.873288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb46e8-223a-4a8e-852d-35e1929c0b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:52.874178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb46e8-223a-4a8e-852d-35e1929c0b53 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:52.874452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb46e8-223a-4a8e-852d-35e1929c0b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:52.874712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb46e8-223a-4a8e-852d-35e1929c0b53 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:52.876057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb46e8-223a-4a8e-852d-35e1929c0b53 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:52.877545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb46e8-223a-4a8e-852d-35e1929c0b53 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:52.882424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 197/409] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:52 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:52.888865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb46e8-223a-4a8e-852d-35e1929c0b53 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:52.944137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369b829-f624-420c-9fc2-553aa0104a38 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:52.987728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d8e29-4a56-4a20-b506-6401ec30bf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:52.988567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d8e29-4a56-4a20-b506-6401ec30bf4a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:52.988855 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d8e29-4a56-4a20-b506-6401ec30bf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:52.989126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d8e29-4a56-4a20-b506-6401ec30bf4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:52.999597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d8e29-4a56-4a20-b506-6401ec30bf4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:53.007051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb46e8-223a-4a8e-852d-35e1929c0b53 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-361d8e29-4a56-4a20-b506-6401ec30bf4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:53.008536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:53.010728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:53.012749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 213/410] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:29:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:53.019773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d8e29-4a56-4a20-b506-6401ec30bf4a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:53.044755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8091a7-bfc2-404e-8ce2-976c1c9fd5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:53.045614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8091a7-bfc2-404e-8ce2-976c1c9fd5a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:53.045886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8091a7-bfc2-404e-8ce2-976c1c9fd5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:53.046207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8091a7-bfc2-404e-8ce2-976c1c9fd5a9 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0/groups/4286f4b1241146f0a49d724fe365a5bf/roles/ec08a86158a44ee2b82951b44b228df2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:53.049650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8091a7-bfc2-404e-8ce2-976c1c9fd5a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=24cf0228dd084700bf349d5310285eb0, group_id=4286f4b1241146f0a49d724fe365a5bf, role_id=ec08a86158a44ee2b82951b44b228df2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:53.051088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8091a7-bfc2-404e-8ce2-976c1c9fd5a9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:53.087040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 198/411] 10.222.0.50 () {58 vars in 1442 bytes} [Fri Apr 29 18:29:52 2022] PUT /identity/v3/projects/24cf0228dd084700bf349d5310285eb0/groups/4286f4b1241146f0a49d724fe365a5bf/roles/ec08a86158a44ee2b82951b44b228df2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:53.127165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d8e29-4a56-4a20-b506-6401ec30bf4a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:53.165389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db927612-b4b3-4243-8082-816e77e3532f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:53.166093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db927612-b4b3-4243-8082-816e77e3532f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:53.166552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db927612-b4b3-4243-8082-816e77e3532f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:53.166841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db927612-b4b3-4243-8082-816e77e3532f None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:53.167601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db927612-b4b3-4243-8082-816e77e3532f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:53.167904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db927612-b4b3-4243-8082-816e77e3532f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:53.169334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db927612-b4b3-4243-8082-816e77e3532f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:53.178833 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 214/412] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:53.185754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8091a7-bfc2-404e-8ce2-976c1c9fd5a9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:53.307571 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8091a7-bfc2-404e-8ce2-976c1c9fd5a9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:53.350813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa53784-701e-413b-b9d7-70f6a651e2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:53.351698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa53784-701e-413b-b9d7-70f6a651e2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:53.351974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa53784-701e-413b-b9d7-70f6a651e2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:53.352244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa53784-701e-413b-b9d7-70f6a651e2f9 None admin] PATH_INFO: `/v3/groups/867b42672a0f4f0c9268fbbd67a6368b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:53.355903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa53784-701e-413b-b9d7-70f6a651e2f9 None admin] RBAC: Authorizing `identity:get_group(group_id=867b42672a0f4f0c9268fbbd67a6368b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:53.357458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa53784-701e-413b-b9d7-70f6a651e2f9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:53.359230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 199/413] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:53 2022] GET /identity/v3/groups/867b42672a0f4f0c9268fbbd67a6368b => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:53.365227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db927612-b4b3-4243-8082-816e77e3532f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:53.484864 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db927612-b4b3-4243-8082-816e77e3532f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:53.522097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167674a9-c014-43e9-b22d-987dc0a837a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:53.522974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167674a9-c014-43e9-b22d-987dc0a837a6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:53.523263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167674a9-c014-43e9-b22d-987dc0a837a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:53.523533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167674a9-c014-43e9-b22d-987dc0a837a6 None admin] PATH_INFO: `/v3/projects/28300f353c4c4c27a229238b2a238c8e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:53.525662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167674a9-c014-43e9-b22d-987dc0a837a6 None admin] RBAC: Authorizing `identity:get_project(project_id=28300f353c4c4c27a229238b2a238c8e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:53.527118 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167674a9-c014-43e9-b22d-987dc0a837a6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:53.529075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 215/414] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:53 2022] GET /identity/v3/projects/28300f353c4c4c27a229238b2a238c8e => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:53.534652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa53784-701e-413b-b9d7-70f6a651e2f9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:53.670204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa53784-701e-413b-b9d7-70f6a651e2f9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:53.710830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6135f-d002-4612-b6b3-b8c99c038795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:53.711775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6135f-d002-4612-b6b3-b8c99c038795 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:53.712407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6135f-d002-4612-b6b3-b8c99c038795 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:53.712830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6135f-d002-4612-b6b3-b8c99c038795 None admin] PATH_INFO: `/v3/projects/28300f353c4c4c27a229238b2a238c8e/groups/867b42672a0f4f0c9268fbbd67a6368b/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:53.717840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6135f-d002-4612-b6b3-b8c99c038795 None admin] RBAC: Authorizing `identity:create_grant(project_id=28300f353c4c4c27a229238b2a238c8e, group_id=867b42672a0f4f0c9268fbbd67a6368b, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:53.719718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6135f-d002-4612-b6b3-b8c99c038795 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:53.742225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 200/415] 10.222.0.50 () {58 vars in 1442 bytes} [Fri Apr 29 18:29:53 2022] PUT /identity/v3/projects/28300f353c4c4c27a229238b2a238c8e/groups/867b42672a0f4f0c9268fbbd67a6368b/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:29:54.122382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bd5c9-6c38-4b09-a18c-6b3ddff48562 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:54.122765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bd5c9-6c38-4b09-a18c-6b3ddff48562 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:54.122929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bd5c9-6c38-4b09-a18c-6b3ddff48562 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:54.124531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 216/416] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:54.130391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e11e9-bd36-4ecf-b97e-cc36175d95e0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:54.130957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e11e9-bd36-4ecf-b97e-cc36175d95e0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:54.131419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e11e9-bd36-4ecf-b97e-cc36175d95e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:54.150388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b03e11e9-bd36-4ecf-b97e-cc36175d95e0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:54.151920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b03e11e9-bd36-4ecf-b97e-cc36175d95e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:54.231382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 201/417] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:29:54.259425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bd5c9-6c38-4b09-a18c-6b3ddff48562 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:54.367845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bd5c9-6c38-4b09-a18c-6b3ddff48562 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:54.415427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e4724-64f8-43f9-b4d0-bb1067b1cb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:54.416227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e4724-64f8-43f9-b4d0-bb1067b1cb4a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:54.416524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e4724-64f8-43f9-b4d0-bb1067b1cb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:54.416847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e4724-64f8-43f9-b4d0-bb1067b1cb4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:54.427579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e4724-64f8-43f9-b4d0-bb1067b1cb4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-390e4724-64f8-43f9-b4d0-bb1067b1cb4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:54.435724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:54.437354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:54.438719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 217/418] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:54.446178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e11e9-bd36-4ecf-b97e-cc36175d95e0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:54.596525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e11e9-bd36-4ecf-b97e-cc36175d95e0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:54.664739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00931ed-1a74-41d8-9993-b9f481068243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:54.665565 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00931ed-1a74-41d8-9993-b9f481068243 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:54.665785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00931ed-1a74-41d8-9993-b9f481068243 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:54.666058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00931ed-1a74-41d8-9993-b9f481068243 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:54.666818 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00931ed-1a74-41d8-9993-b9f481068243 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:54.667128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00931ed-1a74-41d8-9993-b9f481068243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:54.668544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00931ed-1a74-41d8-9993-b9f481068243 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:54.681994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 202/419] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:54.687974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e4724-64f8-43f9-b4d0-bb1067b1cb4a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:54.876878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e4724-64f8-43f9-b4d0-bb1067b1cb4a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:54.925309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bca53d-5b5d-421b-bac1-30e2b3b931cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:54.925876 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bca53d-5b5d-421b-bac1-30e2b3b931cf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:54.926154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bca53d-5b5d-421b-bac1-30e2b3b931cf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:54.926426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bca53d-5b5d-421b-bac1-30e2b3b931cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:55.018583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bca53d-5b5d-421b-bac1-30e2b3b931cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:55.020131 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bca53d-5b5d-421b-bac1-30e2b3b931cf None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:55.106169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879d2f3-aca7-48e4-9394-1bfbb85af812 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:55.106462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879d2f3-aca7-48e4-9394-1bfbb85af812 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:55.106967 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879d2f3-aca7-48e4-9394-1bfbb85af812 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:55.108864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 203/420] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:55.115516 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9d0e2-e833-4f88-8c4b-d6014f1cb01e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:55.115801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9d0e2-e833-4f88-8c4b-d6014f1cb01e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:55.116074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9d0e2-e833-4f88-8c4b-d6014f1cb01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:55.129531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 218/421] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:55.136103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bca53d-5b5d-421b-bac1-30e2b3b931cf None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:55.153532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ada9d0e2-e833-4f88-8c4b-d6014f1cb01e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:55.155492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ada9d0e2-e833-4f88-8c4b-d6014f1cb01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:55.253050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bca53d-5b5d-421b-bac1-30e2b3b931cf None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:55.288473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 204/422] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:55.305500 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb7a8d-3b7d-428d-9df9-f9c2db2f1fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:55.306510 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb7a8d-3b7d-428d-9df9-f9c2db2f1fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:55.306785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb7a8d-3b7d-428d-9df9-f9c2db2f1fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:55.307049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb7a8d-3b7d-428d-9df9-f9c2db2f1fa8 None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:55.308381 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb7a8d-3b7d-428d-9df9-f9c2db2f1fa8 None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:55.309958 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb7a8d-3b7d-428d-9df9-f9c2db2f1fa8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:55.312355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 219/423] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:55 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:55.317009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9d0e2-e833-4f88-8c4b-d6014f1cb01e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:55.319769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb7a8d-3b7d-428d-9df9-f9c2db2f1fa8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:55.459806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb7a8d-3b7d-428d-9df9-f9c2db2f1fa8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:55.496404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2ae05-e1a6-44ee-a418-ead00d9d5b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:55.497743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2ae05-e1a6-44ee-a418-ead00d9d5b3e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:55.498240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2ae05-e1a6-44ee-a418-ead00d9d5b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:55.498754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2ae05-e1a6-44ee-a418-ead00d9d5b3e None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:55.500154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2ae05-e1a6-44ee-a418-ead00d9d5b3e None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:55.501832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2ae05-e1a6-44ee-a418-ead00d9d5b3e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:55.504061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 220/424] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:55 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:55.508587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9d0e2-e833-4f88-8c4b-d6014f1cb01e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:55.510870 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2ae05-e1a6-44ee-a418-ead00d9d5b3e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:55.557568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6650860-899f-4262-a6ec-8bdc25ca75bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:55.558373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6650860-899f-4262-a6ec-8bdc25ca75bf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:55.558649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6650860-899f-4262-a6ec-8bdc25ca75bf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:55.558916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6650860-899f-4262-a6ec-8bdc25ca75bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:55.569005 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6650860-899f-4262-a6ec-8bdc25ca75bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d6650860-899f-4262-a6ec-8bdc25ca75bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:55.576741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:55.577656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:55.578462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 205/425] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:55.579722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7543b97-42c5-47ec-9df3-fe6c7fa0964f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:55.579722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7543b97-42c5-47ec-9df3-fe6c7fa0964f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:55.579981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7543b97-42c5-47ec-9df3-fe6c7fa0964f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:55.586110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 206/426] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:55.587625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7543b97-42c5-47ec-9df3-fe6c7fa0964f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:55.626462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2ae05-e1a6-44ee-a418-ead00d9d5b3e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:55.659147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972af86-1844-4bdd-832a-c1f3f0c65779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:55.659789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972af86-1844-4bdd-832a-c1f3f0c65779 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:55.660069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972af86-1844-4bdd-832a-c1f3f0c65779 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:55.660341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972af86-1844-4bdd-832a-c1f3f0c65779 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:55.661206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972af86-1844-4bdd-832a-c1f3f0c65779 None admin] RBAC: Adding query filter params (group.id=4286f4b1241146f0a49d724fe365a5bf, role.id=3385b41b990e4c6b88d77260a7822c7e, scope.project.id=7a48da3b90914e098e862c1aadf1620b) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:55.661520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972af86-1844-4bdd-832a-c1f3f0c65779 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:55.662936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972af86-1844-4bdd-832a-c1f3f0c65779 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:55.672546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 221/427] 10.222.0.50 () {56 vars in 1376 bytes} [Fri Apr 29 18:29:55 2022] GET /identity/v3/role_assignments?group.id=4286f4b1241146f0a49d724fe365a5bf&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:55.672546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2dac8-5a9f-4045-9f50-f34963af7844 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:55.672546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2dac8-5a9f-4045-9f50-f34963af7844 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:55.672546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2dac8-5a9f-4045-9f50-f34963af7844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:55.699249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ebf2dac8-5a9f-4045-9f50-f34963af7844 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:55.701192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ebf2dac8-5a9f-4045-9f50-f34963af7844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:55.759760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7543b97-42c5-47ec-9df3-fe6c7fa0964f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:55.815467 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 222/428] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:55.828203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fb1ea-4d38-430f-8775-cdaa3fa439eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:55.829153 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fb1ea-4d38-430f-8775-cdaa3fa439eb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:55.829566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fb1ea-4d38-430f-8775-cdaa3fa439eb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:55.830000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fb1ea-4d38-430f-8775-cdaa3fa439eb None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:55.830888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fb1ea-4d38-430f-8775-cdaa3fa439eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:55.831330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fb1ea-4d38-430f-8775-cdaa3fa439eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:55.832878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fb1ea-4d38-430f-8775-cdaa3fa439eb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:55.844777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 207/429] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:29:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:55.850611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2dac8-5a9f-4045-9f50-f34963af7844 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:55.855617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fb1ea-4d38-430f-8775-cdaa3fa439eb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:55.960715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2dac8-5a9f-4045-9f50-f34963af7844 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:55.995035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b760aa-1422-4e19-929b-27a5c7201fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:55.996061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b760aa-1422-4e19-929b-27a5c7201fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:55.996641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b760aa-1422-4e19-929b-27a5c7201fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:55.997034 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fb1ea-4d38-430f-8775-cdaa3fa439eb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:55.997305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b760aa-1422-4e19-929b-27a5c7201fb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.031786 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e0ca8-c61e-4d80-995b-80e388ffbcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:56.031973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e0ca8-c61e-4d80-995b-80e388ffbcfd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.032214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e0ca8-c61e-4d80-995b-80e388ffbcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.032402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e0ca8-c61e-4d80-995b-80e388ffbcfd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.043260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e0ca8-c61e-4d80-995b-80e388ffbcfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b96e0ca8-c61e-4d80-995b-80e388ffbcfd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:56.051416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:56.052373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:29:56.053206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 208/430] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:29:56.061737 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e0ca8-c61e-4d80-995b-80e388ffbcfd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:56.082819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b760aa-1422-4e19-929b-27a5c7201fb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:56.084775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b760aa-1422-4e19-929b-27a5c7201fb0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:56.180839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e0ca8-c61e-4d80-995b-80e388ffbcfd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:56.214909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf868a-e463-483e-9f13-eb67ad52da98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:56.215585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf868a-e463-483e-9f13-eb67ad52da98 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.215864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf868a-e463-483e-9f13-eb67ad52da98 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.216314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf868a-e463-483e-9f13-eb67ad52da98 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.216413 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 223/431] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:56.217655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf868a-e463-483e-9f13-eb67ad52da98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:56.217655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf868a-e463-483e-9f13-eb67ad52da98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:56.218976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf868a-e463-483e-9f13-eb67ad52da98 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:56.220010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72281db3-6586-4f3f-946d-61dac01abe7f None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.220300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72281db3-6586-4f3f-946d-61dac01abe7f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.220599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72281db3-6586-4f3f-946d-61dac01abe7f None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.221889 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 224/432] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:56.227584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72281db3-6586-4f3f-946d-61dac01abe7f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:56.228837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 209/433] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:56.230267 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d37861-31a5-4c1c-a0a5-8f931c692e77 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.230767 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d37861-31a5-4c1c-a0a5-8f931c692e77 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.231043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d37861-31a5-4c1c-a0a5-8f931c692e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.255161 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-94d37861-31a5-4c1c-a0a5-8f931c692e77 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:56.256341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-94d37861-31a5-4c1c-a0a5-8f931c692e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:56.344595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 210/434] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:56.344595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d37861-31a5-4c1c-a0a5-8f931c692e77 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:56.348734 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72281db3-6586-4f3f-946d-61dac01abe7f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:56.388522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df247a-ca68-4e88-b5f8-46785c5ee411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:56.389390 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df247a-ca68-4e88-b5f8-46785c5ee411 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.389670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df247a-ca68-4e88-b5f8-46785c5ee411 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.389940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df247a-ca68-4e88-b5f8-46785c5ee411 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.391101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df247a-ca68-4e88-b5f8-46785c5ee411 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:56.393603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df247a-ca68-4e88-b5f8-46785c5ee411 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:56.394230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 225/435] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:56 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:56.398214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df247a-ca68-4e88-b5f8-46785c5ee411 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:56.455423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d37861-31a5-4c1c-a0a5-8f931c692e77 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:56.492810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df247a-ca68-4e88-b5f8-46785c5ee411 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:56.497029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0674d6-f648-4f37-8b32-50b892694414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:56.497696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0674d6-f648-4f37-8b32-50b892694414 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.497970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0674d6-f648-4f37-8b32-50b892694414 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.498263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0674d6-f648-4f37-8b32-50b892694414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.527543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b569e1d-04a0-45ef-b5f3-d9b254a185be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:56.528376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b569e1d-04a0-45ef-b5f3-d9b254a185be None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.528685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b569e1d-04a0-45ef-b5f3-d9b254a185be None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.528955 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b569e1d-04a0-45ef-b5f3-d9b254a185be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.540826 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b569e1d-04a0-45ef-b5f3-d9b254a185be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0b569e1d-04a0-45ef-b5f3-d9b254a185be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:56.548943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:56.550088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:56.550898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 226/436] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:29:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:56.551560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b569e1d-04a0-45ef-b5f3-d9b254a185be None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:56.575926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0674d6-f648-4f37-8b32-50b892694414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:56.578040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0674d6-f648-4f37-8b32-50b892694414 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:56.655981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b569e1d-04a0-45ef-b5f3-d9b254a185be None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:56.681547 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 211/437] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:56.681547 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0674d6-f648-4f37-8b32-50b892694414 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:56.695170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0c578-ccc4-40c6-aa38-219b42bdc577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:56.696068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0c578-ccc4-40c6-aa38-219b42bdc577 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.696540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0c578-ccc4-40c6-aa38-219b42bdc577 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.696813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0c578-ccc4-40c6-aa38-219b42bdc577 None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.698034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0c578-ccc4-40c6-aa38-219b42bdc577 None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:56.699465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0c578-ccc4-40c6-aa38-219b42bdc577 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:56.706033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 227/438] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:56 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:56.706033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0c578-ccc4-40c6-aa38-219b42bdc577 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:56.779182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0674d6-f648-4f37-8b32-50b892694414 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:56.810346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0c578-ccc4-40c6-aa38-219b42bdc577 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:56.813938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33563f56-adc5-4df0-a430-5e069c6654b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:56.814693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33563f56-adc5-4df0-a430-5e069c6654b3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.814976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33563f56-adc5-4df0-a430-5e069c6654b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.815250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33563f56-adc5-4df0-a430-5e069c6654b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.815988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33563f56-adc5-4df0-a430-5e069c6654b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:56.816294 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33563f56-adc5-4df0-a430-5e069c6654b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:56.817738 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33563f56-adc5-4df0-a430-5e069c6654b3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:56.826831 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 212/439] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:29:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:56.828123 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e199ae-7bd1-4664-8882-36da7dc8200d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.828416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e199ae-7bd1-4664-8882-36da7dc8200d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.828719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e199ae-7bd1-4664-8882-36da7dc8200d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.830085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 213/440] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:29:56.830662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e199ae-7bd1-4664-8882-36da7dc8200d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:56.846634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398ba3a-61a3-4ff3-9aaf-1c975a3c7e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:56.847499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398ba3a-61a3-4ff3-9aaf-1c975a3c7e3f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.847774 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398ba3a-61a3-4ff3-9aaf-1c975a3c7e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.848042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398ba3a-61a3-4ff3-9aaf-1c975a3c7e3f None admin] PATH_INFO: `/v3/projects/7a48da3b90914e098e862c1aadf1620b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.849254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398ba3a-61a3-4ff3-9aaf-1c975a3c7e3f None admin] RBAC: Authorizing `identity:get_project(project_id=7a48da3b90914e098e862c1aadf1620b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:56.850697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398ba3a-61a3-4ff3-9aaf-1c975a3c7e3f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:56.852353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 228/441] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:56 2022] GET /identity/v3/projects/7a48da3b90914e098e862c1aadf1620b => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:56.856952 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398ba3a-61a3-4ff3-9aaf-1c975a3c7e3f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:56.928993 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e199ae-7bd1-4664-8882-36da7dc8200d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:56.947594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398ba3a-61a3-4ff3-9aaf-1c975a3c7e3f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:56.960165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d0585-1ed2-4944-af22-f5ef5b88e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:56.960838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d0585-1ed2-4944-af22-f5ef5b88e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.961115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d0585-1ed2-4944-af22-f5ef5b88e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.961382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d0585-1ed2-4944-af22-f5ef5b88e7f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.962298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d0585-1ed2-4944-af22-f5ef5b88e7f8 None admin] RBAC: Adding query filter params (group.id=4286f4b1241146f0a49d724fe365a5bf, role.id=3385b41b990e4c6b88d77260a7822c7e, scope.project.id=24cf0228dd084700bf349d5310285eb0) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:56.962613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d0585-1ed2-4944-af22-f5ef5b88e7f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:56.964214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d0585-1ed2-4944-af22-f5ef5b88e7f8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:56.970196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 214/442] 10.222.0.50 () {56 vars in 1376 bytes} [Fri Apr 29 18:29:56 2022] GET /identity/v3/role_assignments?group.id=4286f4b1241146f0a49d724fe365a5bf&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=3385b41b990e4c6b88d77260a7822c7e => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:56.971321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290a65a-ce1c-4c83-8634-655441134c7d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.971600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290a65a-ce1c-4c83-8634-655441134c7d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.971866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290a65a-ce1c-4c83-8634-655441134c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:56.989656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2290a65a-ce1c-4c83-8634-655441134c7d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:29:56.990872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2290a65a-ce1c-4c83-8634-655441134c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:29:56.992641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087f802-065d-4b6b-9840-8d9f42b5ebdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:56.993427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087f802-065d-4b6b-9840-8d9f42b5ebdf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:56.993660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087f802-065d-4b6b-9840-8d9f42b5ebdf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:56.993913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087f802-065d-4b6b-9840-8d9f42b5ebdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:57.063965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 215/443] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:29:57.065507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290a65a-ce1c-4c83-8634-655441134c7d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:57.074435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087f802-065d-4b6b-9840-8d9f42b5ebdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:57.076878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087f802-065d-4b6b-9840-8d9f42b5ebdf None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:57.178232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290a65a-ce1c-4c83-8634-655441134c7d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:57.181672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 229/444] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:57.187955 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087f802-065d-4b6b-9840-8d9f42b5ebdf None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:57.218995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54735692-60ea-4c64-9cc2-e50beaa17875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:57.219863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54735692-60ea-4c64-9cc2-e50beaa17875 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:57.220135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54735692-60ea-4c64-9cc2-e50beaa17875 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:57.220406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54735692-60ea-4c64-9cc2-e50beaa17875 None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:57.221612 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54735692-60ea-4c64-9cc2-e50beaa17875 None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:57.223047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54735692-60ea-4c64-9cc2-e50beaa17875 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:57.224885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 216/445] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:57 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:57.225430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54735692-60ea-4c64-9cc2-e50beaa17875 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:57.278415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087f802-065d-4b6b-9840-8d9f42b5ebdf None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:57.311537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5aec89-6543-4076-a445-14bc01c825c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:57.312326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5aec89-6543-4076-a445-14bc01c825c5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:57.312623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5aec89-6543-4076-a445-14bc01c825c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:57.312894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5aec89-6543-4076-a445-14bc01c825c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:57.318232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54735692-60ea-4c64-9cc2-e50beaa17875 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:57.323186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5aec89-6543-4076-a445-14bc01c825c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-fd5aec89-6543-4076-a445-14bc01c825c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:29:57.330722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:29:57.331587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:29:57.332399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 230/446] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:29:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:29:57.333732 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5aec89-6543-4076-a445-14bc01c825c5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:57.357154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19972c6-1d18-408f-b145-ad98d3874541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:57.358011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19972c6-1d18-408f-b145-ad98d3874541 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:57.358291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19972c6-1d18-408f-b145-ad98d3874541 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:57.358566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19972c6-1d18-408f-b145-ad98d3874541 None admin] PATH_INFO: `/v3/projects/24cf0228dd084700bf349d5310285eb0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:57.359745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19972c6-1d18-408f-b145-ad98d3874541 None admin] RBAC: Authorizing `identity:get_project(project_id=24cf0228dd084700bf349d5310285eb0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:57.361210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19972c6-1d18-408f-b145-ad98d3874541 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:57.362905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 217/447] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:57 2022] GET /identity/v3/projects/24cf0228dd084700bf349d5310285eb0 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:57.363643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19972c6-1d18-408f-b145-ad98d3874541 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:57.434840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5aec89-6543-4076-a445-14bc01c825c5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:57.461120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19972c6-1d18-408f-b145-ad98d3874541 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:57.471150 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b09a8-66e8-414b-b04c-ee3fa86cfa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:57.471804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b09a8-66e8-414b-b04c-ee3fa86cfa06 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:57.472082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b09a8-66e8-414b-b04c-ee3fa86cfa06 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:57.472350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b09a8-66e8-414b-b04c-ee3fa86cfa06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:57.473200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b09a8-66e8-414b-b04c-ee3fa86cfa06 None admin] RBAC: Adding query filter params (group.id=4286f4b1241146f0a49d724fe365a5bf, role.id=ec08a86158a44ee2b82951b44b228df2, scope.project.id=7a48da3b90914e098e862c1aadf1620b) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:57.473509 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b09a8-66e8-414b-b04c-ee3fa86cfa06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:57.474930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b09a8-66e8-414b-b04c-ee3fa86cfa06 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:57.480849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 231/448] 10.222.0.50 () {56 vars in 1376 bytes} [Fri Apr 29 18:29:57 2022] GET /identity/v3/role_assignments?group.id=4286f4b1241146f0a49d724fe365a5bf&scope.project.id=7a48da3b90914e098e862c1aadf1620b&role.id=ec08a86158a44ee2b82951b44b228df2 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:57.481596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b09a8-66e8-414b-b04c-ee3fa86cfa06 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:57.494939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae973ad6-8ead-4bbc-a5fa-6bb642d5119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:57.495918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae973ad6-8ead-4bbc-a5fa-6bb642d5119a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:57.496195 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae973ad6-8ead-4bbc-a5fa-6bb642d5119a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:57.496462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae973ad6-8ead-4bbc-a5fa-6bb642d5119a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:57.497446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae973ad6-8ead-4bbc-a5fa-6bb642d5119a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:57.498210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae973ad6-8ead-4bbc-a5fa-6bb642d5119a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:57.499923 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae973ad6-8ead-4bbc-a5fa-6bb642d5119a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:57.516770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 218/449] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:29:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:57.518734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae973ad6-8ead-4bbc-a5fa-6bb642d5119a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:57.586237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b09a8-66e8-414b-b04c-ee3fa86cfa06 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:57.628677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d887e-63f7-4812-ba96-aabd24534630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:57.629570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d887e-63f7-4812-ba96-aabd24534630 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:57.629847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d887e-63f7-4812-ba96-aabd24534630 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:57.630117 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d887e-63f7-4812-ba96-aabd24534630 None admin] PATH_INFO: `/v3/groups/4286f4b1241146f0a49d724fe365a5bf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:57.631830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d887e-63f7-4812-ba96-aabd24534630 None admin] RBAC: Authorizing `identity:get_group(group_id=4286f4b1241146f0a49d724fe365a5bf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:57.633469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d887e-63f7-4812-ba96-aabd24534630 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:57.635203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 232/450] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:57 2022] GET /identity/v3/groups/4286f4b1241146f0a49d724fe365a5bf => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:57.641282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d887e-63f7-4812-ba96-aabd24534630 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:57.643834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae973ad6-8ead-4bbc-a5fa-6bb642d5119a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:57.684522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113586c-1948-4498-a860-595aece683a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:57.685574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113586c-1948-4498-a860-595aece683a2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:57.685574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113586c-1948-4498-a860-595aece683a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:57.685803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113586c-1948-4498-a860-595aece683a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:57.758639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d887e-63f7-4812-ba96-aabd24534630 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:57.766166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113586c-1948-4498-a860-595aece683a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:57.767696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113586c-1948-4498-a860-595aece683a2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:57.802507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc6e3c-d0b0-416f-ab3f-056aa1acf178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:57.803173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc6e3c-d0b0-416f-ab3f-056aa1acf178 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:57.803451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc6e3c-d0b0-416f-ab3f-056aa1acf178 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:57.803749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc6e3c-d0b0-416f-ab3f-056aa1acf178 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:57.805843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc6e3c-d0b0-416f-ab3f-056aa1acf178 None admin] RBAC: Adding query filter params (group.id=4286f4b1241146f0a49d724fe365a5bf, role.id=ec08a86158a44ee2b82951b44b228df2, scope.project.id=24cf0228dd084700bf349d5310285eb0) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:57.806201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc6e3c-d0b0-416f-ab3f-056aa1acf178 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:57.807649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc6e3c-d0b0-416f-ab3f-056aa1acf178 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:57.813998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 233/451] 10.222.0.50 () {56 vars in 1376 bytes} [Fri Apr 29 18:29:57 2022] GET /identity/v3/role_assignments?group.id=4286f4b1241146f0a49d724fe365a5bf&scope.project.id=24cf0228dd084700bf349d5310285eb0&role.id=ec08a86158a44ee2b82951b44b228df2 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:57.874598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 219/452] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:29:57.880750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc6e3c-d0b0-416f-ab3f-056aa1acf178 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:57.992839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc6e3c-d0b0-416f-ab3f-056aa1acf178 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:58.026817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe4b03-1391-4a4e-8e2a-7f82f7673a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:58.027797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe4b03-1391-4a4e-8e2a-7f82f7673a06 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:58.027975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe4b03-1391-4a4e-8e2a-7f82f7673a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:58.028240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe4b03-1391-4a4e-8e2a-7f82f7673a06 None admin] PATH_INFO: `/v3/projects/28300f353c4c4c27a229238b2a238c8e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:58.029447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe4b03-1391-4a4e-8e2a-7f82f7673a06 None admin] RBAC: Authorizing `identity:get_project(project_id=28300f353c4c4c27a229238b2a238c8e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:58.030899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe4b03-1391-4a4e-8e2a-7f82f7673a06 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:58.033019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 234/453] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:29:57 2022] GET /identity/v3/projects/28300f353c4c4c27a229238b2a238c8e => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:29:58.038662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113586c-1948-4498-a860-595aece683a2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:58.141968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113586c-1948-4498-a860-595aece683a2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:58.175237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2c24b-ed75-45de-b7b9-c2a7f928f4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:58.176103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2c24b-ed75-45de-b7b9-c2a7f928f4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:58.176374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2c24b-ed75-45de-b7b9-c2a7f928f4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:58.176663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2c24b-ed75-45de-b7b9-c2a7f928f4f0 None admin] PATH_INFO: `/v3/groups/867b42672a0f4f0c9268fbbd67a6368b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:58.177816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2c24b-ed75-45de-b7b9-c2a7f928f4f0 None admin] RBAC: Authorizing `identity:get_group(group_id=867b42672a0f4f0c9268fbbd67a6368b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:58.179246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2c24b-ed75-45de-b7b9-c2a7f928f4f0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:58.180958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 220/454] 10.222.0.50 () {56 vars in 1180 bytes} [Fri Apr 29 18:29:58 2022] GET /identity/v3/groups/867b42672a0f4f0c9268fbbd67a6368b => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:29:58.187120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe4b03-1391-4a4e-8e2a-7f82f7673a06 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:29:58.285983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe4b03-1391-4a4e-8e2a-7f82f7673a06 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:29:58.319153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7cc1c-0b04-42e6-8a19-867a52442aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:29:58.319795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7cc1c-0b04-42e6-8a19-867a52442aad None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:29:58.320071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7cc1c-0b04-42e6-8a19-867a52442aad None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:29:58.320334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7cc1c-0b04-42e6-8a19-867a52442aad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:29:58.321177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7cc1c-0b04-42e6-8a19-867a52442aad None admin] RBAC: Adding query filter params (group.id=867b42672a0f4f0c9268fbbd67a6368b, role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=28300f353c4c4c27a229238b2a238c8e) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:29:58.321559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7cc1c-0b04-42e6-8a19-867a52442aad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:29:58.323163 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7cc1c-0b04-42e6-8a19-867a52442aad None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:29:58.328922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 235/455] 10.222.0.50 () {56 vars in 1376 bytes} [Fri Apr 29 18:29:58 2022] GET /identity/v3/role_assignments?group.id=867b42672a0f4f0c9268fbbd67a6368b&scope.project.id=28300f353c4c4c27a229238b2a238c8e&role.id=e55ef11844a742f5889243f189bd59bc => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:00.992070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953f0ee-1e82-466a-8412-f1c0b064e074 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:00.992623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953f0ee-1e82-466a-8412-f1c0b064e074 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:00.992623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953f0ee-1e82-466a-8412-f1c0b064e074 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:00.994560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 221/456] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:01.000080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5d878-b04c-4aa1-b188-b83dc45c2290 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.000510 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5d878-b04c-4aa1-b188-b83dc45c2290 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.000677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5d878-b04c-4aa1-b188-b83dc45c2290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.020708 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-afa5d878-b04c-4aa1-b188-b83dc45c2290 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:01.021893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-afa5d878-b04c-4aa1-b188-b83dc45c2290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:01.111139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 236/457] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:30:01.147681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953f0ee-1e82-466a-8412-f1c0b064e074 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:01.293179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953f0ee-1e82-466a-8412-f1c0b064e074 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:01.328834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc03373-412f-485a-af49-5be321255baf None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.328834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc03373-412f-485a-af49-5be321255baf None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.328834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc03373-412f-485a-af49-5be321255baf None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.330966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 237/458] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:01.332971 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e1675-974f-4abf-910d-641fc9d3143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:01.334590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e1675-974f-4abf-910d-641fc9d3143a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.335305 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e1675-974f-4abf-910d-641fc9d3143a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.335977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e1675-974f-4abf-910d-641fc9d3143a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.343448 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a406f-198e-4c10-810a-2b21037db5d8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.344090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a406f-198e-4c10-810a-2b21037db5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.344630 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a406f-198e-4c10-810a-2b21037db5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.351840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e1675-974f-4abf-910d-641fc9d3143a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:01.353629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e1675-974f-4abf-910d-641fc9d3143a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f50e1675-974f-4abf-910d-641fc9d3143a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:01.373716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:01.374954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:01.375829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:01.375829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:01.375829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:01.375829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:01.375829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 222/459] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:01.381158 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e1675-974f-4abf-910d-641fc9d3143a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:01.390512 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-2c8a406f-198e-4c10-810a-2b21037db5d8 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:01.392004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2c8a406f-198e-4c10-810a-2b21037db5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:01.499127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e1675-974f-4abf-910d-641fc9d3143a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:01.515559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 238/460] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:30:01.541366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0f17-d3f6-464e-b1c0-ef5904e49021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:01.542045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0f17-d3f6-464e-b1c0-ef5904e49021 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.542324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0f17-d3f6-464e-b1c0-ef5904e49021 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.542593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0f17-d3f6-464e-b1c0-ef5904e49021 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.543370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a0f17-d3f6-464e-b1c0-ef5904e49021 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:01.543614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a0f17-d3f6-464e-b1c0-ef5904e49021 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:01.545012 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a0f17-d3f6-464e-b1c0-ef5904e49021 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:01.555170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a406f-198e-4c10-810a-2b21037db5d8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:01.568732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 223/461] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:01.574711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0f17-d3f6-464e-b1c0-ef5904e49021 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:01.680568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a406f-198e-4c10-810a-2b21037db5d8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:01.685272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0f17-d3f6-464e-b1c0-ef5904e49021 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:01.714387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd886f9-67bf-42ae-a891-5fb5a33749cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:01.715188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd886f9-67bf-42ae-a891-5fb5a33749cf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.715470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd886f9-67bf-42ae-a891-5fb5a33749cf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.715740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd886f9-67bf-42ae-a891-5fb5a33749cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.717025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7e5d4-4be0-4b17-98b9-ce9b38fd7327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:01.718043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7e5d4-4be0-4b17-98b9-ce9b38fd7327 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.718328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7e5d4-4be0-4b17-98b9-ce9b38fd7327 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.718600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7e5d4-4be0-4b17-98b9-ce9b38fd7327 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.719291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7e5d4-4be0-4b17-98b9-ce9b38fd7327 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:01.720736 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7e5d4-4be0-4b17-98b9-ce9b38fd7327 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:01.726187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd886f9-67bf-42ae-a891-5fb5a33749cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:01.727839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd886f9-67bf-42ae-a891-5fb5a33749cf None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:01.748531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 224/462] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:30:01 2022] POST /identity/v3/users => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:01.748531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec40989-6f1c-43fc-81be-78ab738ae610 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.748531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec40989-6f1c-43fc-81be-78ab738ae610 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.748531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec40989-6f1c-43fc-81be-78ab738ae610 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0fd886f9-67bf-42ae-a891-5fb5a33749cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:01.749002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:01.749973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:01.750809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:01.750809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:01.750809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:01.750809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:01.750809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 239/463] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:01.750809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59380a12-94ab-428b-9d5c-7f7e147b2813 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.750809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59380a12-94ab-428b-9d5c-7f7e147b2813 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.751179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59380a12-94ab-428b-9d5c-7f7e147b2813 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.756537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 225/464] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:01 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:01.756733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 240/465] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:01.756825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec40989-6f1c-43fc-81be-78ab738ae610 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:01.756929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c7e59-cebc-462f-aa9e-f7aff41016e6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.756929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c7e59-cebc-462f-aa9e-f7aff41016e6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.756929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c7e59-cebc-462f-aa9e-f7aff41016e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.774344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c82c7e59-cebc-462f-aa9e-f7aff41016e6 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:01.775884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c82c7e59-cebc-462f-aa9e-f7aff41016e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:01.872792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 241/466] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:01.874246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbefac-5976-4ab2-9ec4-3dd0ea91a311 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.874536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbefac-5976-4ab2-9ec4-3dd0ea91a311 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.874806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbefac-5976-4ab2-9ec4-3dd0ea91a311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.878081 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec40989-6f1c-43fc-81be-78ab738ae610 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:01.906473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-64fbefac-5976-4ab2-9ec4-3dd0ea91a311 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:01.907681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-64fbefac-5976-4ab2-9ec4-3dd0ea91a311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:01.933384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854df9a-4e57-4573-b8cd-370f8032cdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:01.934330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854df9a-4e57-4573-b8cd-370f8032cdd8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:01.934605 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854df9a-4e57-4573-b8cd-370f8032cdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:01.935036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854df9a-4e57-4573-b8cd-370f8032cdd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:01.936120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854df9a-4e57-4573-b8cd-370f8032cdd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:01.936709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854df9a-4e57-4573-b8cd-370f8032cdd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:01.938300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854df9a-4e57-4573-b8cd-370f8032cdd8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:01.953561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 226/467] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:01.958145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854df9a-4e57-4573-b8cd-370f8032cdd8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:01.985262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 242/468] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:01.986100 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbefac-5976-4ab2-9ec4-3dd0ea91a311 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:02.064833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854df9a-4e57-4573-b8cd-370f8032cdd8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:02.099490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b6a2e-8806-4904-b0e1-b4aa96fd46fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:02.100291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b6a2e-8806-4904-b0e1-b4aa96fd46fa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:02.100615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b6a2e-8806-4904-b0e1-b4aa96fd46fa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:02.100904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b6a2e-8806-4904-b0e1-b4aa96fd46fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:02.103016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbefac-5976-4ab2-9ec4-3dd0ea91a311 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:02.111393 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b6a2e-8806-4904-b0e1-b4aa96fd46fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:02.112828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b6a2e-8806-4904-b0e1-b4aa96fd46fa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ce3b6a2e-8806-4904-b0e1-b4aa96fd46fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:02.130352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:02.131218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:02.132078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:02.132078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:02.132078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:02.132078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:02.132078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 227/469] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:02.136632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b6a2e-8806-4904-b0e1-b4aa96fd46fa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:02.140573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb9895-7552-4300-80a5-941e7cfb6060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:02.141433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb9895-7552-4300-80a5-941e7cfb6060 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:02.141846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb9895-7552-4300-80a5-941e7cfb6060 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:02.142272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb9895-7552-4300-80a5-941e7cfb6060 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:02.143153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb9895-7552-4300-80a5-941e7cfb6060 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:02.144768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb9895-7552-4300-80a5-941e7cfb6060 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:02.169744 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 243/470] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:30:01 2022] POST /identity/v3/users => generated 248 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:02.176918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb9895-7552-4300-80a5-941e7cfb6060 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:02.239784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b6a2e-8806-4904-b0e1-b4aa96fd46fa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:02.275813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44230-312f-4e25-b95f-f69b52528edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:02.276652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44230-312f-4e25-b95f-f69b52528edc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:02.276935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44230-312f-4e25-b95f-f69b52528edc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:02.277209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44230-312f-4e25-b95f-f69b52528edc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:02.303752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44230-312f-4e25-b95f-f69b52528edc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:02.306572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44230-312f-4e25-b95f-f69b52528edc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:02.342636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb9895-7552-4300-80a5-941e7cfb6060 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d2b44230-312f-4e25-b95f-f69b52528edc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:02.345395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:02.346404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:02.347397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:02.347397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:02.347397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:02.347397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:02.347397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 228/471] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:02.353905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44230-312f-4e25-b95f-f69b52528edc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:02.426284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df99a-a4cf-466a-8e90-0fa87ea1404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:02.426943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df99a-a4cf-466a-8e90-0fa87ea1404e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:02.427225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df99a-a4cf-466a-8e90-0fa87ea1404e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:02.427495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df99a-a4cf-466a-8e90-0fa87ea1404e None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:02.428240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99df99a-a4cf-466a-8e90-0fa87ea1404e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:02.428559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99df99a-a4cf-466a-8e90-0fa87ea1404e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:02.430062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99df99a-a4cf-466a-8e90-0fa87ea1404e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:02.455027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 244/472] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:02.461628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df99a-a4cf-466a-8e90-0fa87ea1404e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:02.510107 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44230-312f-4e25-b95f-f69b52528edc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:02.543884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671f1b9-4ed4-41df-b8b8-cedf552316cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:02.544936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671f1b9-4ed4-41df-b8b8-cedf552316cb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:02.545402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671f1b9-4ed4-41df-b8b8-cedf552316cb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:02.545841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671f1b9-4ed4-41df-b8b8-cedf552316cb None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:02.546728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1671f1b9-4ed4-41df-b8b8-cedf552316cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:02.547212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1671f1b9-4ed4-41df-b8b8-cedf552316cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:02.548787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1671f1b9-4ed4-41df-b8b8-cedf552316cb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:02.560141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df99a-a4cf-466a-8e90-0fa87ea1404e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:02.561295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 229/473] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:02.566855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671f1b9-4ed4-41df-b8b8-cedf552316cb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:02.595669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be2b23-a19a-4118-873e-de74fd354674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:02.596359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be2b23-a19a-4118-873e-de74fd354674 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:02.596822 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be2b23-a19a-4118-873e-de74fd354674 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:02.597619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be2b23-a19a-4118-873e-de74fd354674 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:02.597821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be2b23-a19a-4118-873e-de74fd354674 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:02.599339 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be2b23-a19a-4118-873e-de74fd354674 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:02.623742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 245/474] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:30:02 2022] POST /identity/v3/users => generated 248 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:02.655053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56edd8-d016-47aa-b91b-a4a134fdb248 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:02.655363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56edd8-d016-47aa-b91b-a4a134fdb248 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:02.655599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56edd8-d016-47aa-b91b-a4a134fdb248 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:02.657826 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 246/475] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:30:02.663686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f612ed-5ccd-480d-8826-33981e462809 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:02.664021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f612ed-5ccd-480d-8826-33981e462809 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:02.664301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f612ed-5ccd-480d-8826-33981e462809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:02.702669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-48f612ed-5ccd-480d-8826-33981e462809 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:02.703884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671f1b9-4ed4-41df-b8b8-cedf552316cb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:02.704010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-48f612ed-5ccd-480d-8826-33981e462809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:02.743130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa31c8-3bd8-4639-b8d6-b360b0a3b208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:02.743914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa31c8-3bd8-4639-b8d6-b360b0a3b208 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:02.744198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa31c8-3bd8-4639-b8d6-b360b0a3b208 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:02.744484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa31c8-3bd8-4639-b8d6-b360b0a3b208 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:02.745265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa31c8-3bd8-4639-b8d6-b360b0a3b208 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:02.746841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa31c8-3bd8-4639-b8d6-b360b0a3b208 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:02.783922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 230/476] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:30:02 2022] POST /identity/v3/users => generated 249 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:30:02.795328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 247/477] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:30:02.831163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa31c8-3bd8-4639-b8d6-b360b0a3b208 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:02.952529 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa31c8-3bd8-4639-b8d6-b360b0a3b208 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:02.989089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55500abb-8830-4160-a9a9-a54cb7e54533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:02.989787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55500abb-8830-4160-a9a9-a54cb7e54533 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:02.990071 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55500abb-8830-4160-a9a9-a54cb7e54533 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:02.990341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55500abb-8830-4160-a9a9-a54cb7e54533 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:02.991103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55500abb-8830-4160-a9a9-a54cb7e54533 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:02.992540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55500abb-8830-4160-a9a9-a54cb7e54533 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:03.001263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.sql.core [None req-55500abb-8830-4160-a9a9-a54cb7e54533 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Apr 29 18:30:03.001263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Apr 29 18:30:03.001263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [parameters: {'id': 'a13a7eade26340648b8fa923df7d7b59', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Apr 29 18:30:03.001263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-55500abb-8830-4160-a9a9-a54cb7e54533 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Apr 29 18:30:03.018610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return ref.to_dict() Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Apr 29 18:30:03.019544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application yield resource Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.rollback() Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.force_reraise() Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise self.value Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application session.commit() Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1431, in commit Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Apr 29 18:30:03.020761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._prepare_impl() Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.flush() Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3363, in flush Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._flush(objects) Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3503, in _flush Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application compat.raise_( Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3463, in _flush Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application flush_context.execute() Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rec.execute(self) Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Apr 29 18:30:03.021978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application _emit_insert_statements( Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application c = connection._execute_20( Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self._execute_context( Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Apr 29 18:30:03.022838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Apr 29 18:30:03.023642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [parameters: {'id': 'a13a7eade26340648b8fa923df7d7b59', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:03.024760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Apr 29 18:30:03.026397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:03.027929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 231/478] 10.222.0.50 () {60 vars in 1131 bytes} [Fri Apr 29 18:30:02 2022] POST /identity/v3/roles => generated 144 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:03.028059 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f612ed-5ccd-480d-8826-33981e462809 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:03.141711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f612ed-5ccd-480d-8826-33981e462809 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:03.176543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394121a6-a9d9-4332-a7c8-ce1d8ef970bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:03.177374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394121a6-a9d9-4332-a7c8-ce1d8ef970bc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:03.177655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394121a6-a9d9-4332-a7c8-ce1d8ef970bc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:03.177929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394121a6-a9d9-4332-a7c8-ce1d8ef970bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:03.188412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394121a6-a9d9-4332-a7c8-ce1d8ef970bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-394121a6-a9d9-4332-a7c8-ce1d8ef970bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:03.196684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:03.198786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:03.200712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 248/479] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:30:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:03.203782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55500abb-8830-4160-a9a9-a54cb7e54533 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:03.311797 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55500abb-8830-4160-a9a9-a54cb7e54533 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:03.346536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f1832-0da8-4980-805f-2e8cbe8589c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:03.347504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f1832-0da8-4980-805f-2e8cbe8589c5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:03.347953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f1832-0da8-4980-805f-2e8cbe8589c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:03.348391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f1832-0da8-4980-805f-2e8cbe8589c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:03.349362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f1832-0da8-4980-805f-2e8cbe8589c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:03.349830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f1832-0da8-4980-805f-2e8cbe8589c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:03.351425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f1832-0da8-4980-805f-2e8cbe8589c5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:03.360229 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 232/480] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:04.386065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b9a10-6fc6-40a8-b90b-c637263aed21 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:04.386625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b9a10-6fc6-40a8-b90b-c637263aed21 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:04.386625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b9a10-6fc6-40a8-b90b-c637263aed21 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:04.388261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 249/481] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:04.393352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6aadf-fb7f-495b-89b9-823d1ae53fb5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:04.393745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6aadf-fb7f-495b-89b9-823d1ae53fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:04.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6aadf-fb7f-495b-89b9-823d1ae53fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:04.417499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5cd6aadf-fb7f-495b-89b9-823d1ae53fb5 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:04.418691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5cd6aadf-fb7f-495b-89b9-823d1ae53fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:04.495175 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 233/482] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:04.531096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b9a10-6fc6-40a8-b90b-c637263aed21 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:04.639850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b9a10-6fc6-40a8-b90b-c637263aed21 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:04.680911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8edecc-7c2a-425b-9c71-c412b88f9fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:04.681724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8edecc-7c2a-425b-9c71-c412b88f9fec None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:04.682006 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8edecc-7c2a-425b-9c71-c412b88f9fec None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:04.682276 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8edecc-7c2a-425b-9c71-c412b88f9fec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:04.701355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8edecc-7c2a-425b-9c71-c412b88f9fec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2b8edecc-7c2a-425b-9c71-c412b88f9fec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:04.709242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:04.710141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:04.710927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:04.715090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 250/483] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:04.716410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6aadf-fb7f-495b-89b9-823d1ae53fb5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:04.821390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6aadf-fb7f-495b-89b9-823d1ae53fb5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:04.855744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098f4d8-fbb1-422b-bdfe-e6403a4c4ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:04.856420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098f4d8-fbb1-422b-bdfe-e6403a4c4ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:04.856732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098f4d8-fbb1-422b-bdfe-e6403a4c4ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:04.857008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098f4d8-fbb1-422b-bdfe-e6403a4c4ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:04.857762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098f4d8-fbb1-422b-bdfe-e6403a4c4ed1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:04.858064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098f4d8-fbb1-422b-bdfe-e6403a4c4ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:04.859445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098f4d8-fbb1-422b-bdfe-e6403a4c4ed1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:04.868033 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 234/484] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:04.873420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8edecc-7c2a-425b-9c71-c412b88f9fec None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:04.991916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8edecc-7c2a-425b-9c71-c412b88f9fec None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:05.031556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d2e59-31f5-4ba2-aa73-9ac5ca55777d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:05.032767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d2e59-31f5-4ba2-aa73-9ac5ca55777d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.033262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d2e59-31f5-4ba2-aa73-9ac5ca55777d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.033808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d2e59-31f5-4ba2-aa73-9ac5ca55777d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.051109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d2e59-31f5-4ba2-aa73-9ac5ca55777d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:05.052938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d2e59-31f5-4ba2-aa73-9ac5ca55777d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-613d2e59-31f5-4ba2-aa73-9ac5ca55777d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:05.074474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:05.075400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:05.076335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:05.076335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:05.076335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:05.076335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:05.078312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 251/485] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:05.092343 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098f4d8-fbb1-422b-bdfe-e6403a4c4ed1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:05.143836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f5b5b-e5f5-4f60-825c-f0c5f8d5b40e None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.144152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f5b5b-e5f5-4f60-825c-f0c5f8d5b40e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.144394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f5b5b-e5f5-4f60-825c-f0c5f8d5b40e None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.147054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 252/486] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:05.151306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb5b19-7ac7-4ab1-8ebe-cde38ed88144 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.151590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb5b19-7ac7-4ab1-8ebe-cde38ed88144 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.151865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb5b19-7ac7-4ab1-8ebe-cde38ed88144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.169010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3aeb5b19-7ac7-4ab1-8ebe-cde38ed88144 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:05.170231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3aeb5b19-7ac7-4ab1-8ebe-cde38ed88144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:05.266515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098f4d8-fbb1-422b-bdfe-e6403a4c4ed1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:05.302193 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce5ed5-0e35-433d-bd50-d9c9509711d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:05.302864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce5ed5-0e35-433d-bd50-d9c9509711d7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.303146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce5ed5-0e35-433d-bd50-d9c9509711d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.303421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce5ed5-0e35-433d-bd50-d9c9509711d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.304146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce5ed5-0e35-433d-bd50-d9c9509711d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:05.304452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce5ed5-0e35-433d-bd50-d9c9509711d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:05.305946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce5ed5-0e35-433d-bd50-d9c9509711d7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:05.309324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 253/487] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:30:05.326030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 235/488] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:05.331200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb5b19-7ac7-4ab1-8ebe-cde38ed88144 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:05.344522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce5ed5-0e35-433d-bd50-d9c9509711d7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:05.453136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce5ed5-0e35-433d-bd50-d9c9509711d7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:05.487383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859dd4c-dcc7-411a-bae4-ac5f89cce0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:05.488192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859dd4c-dcc7-411a-bae4-ac5f89cce0fb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.488472 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859dd4c-dcc7-411a-bae4-ac5f89cce0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.488773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859dd4c-dcc7-411a-bae4-ac5f89cce0fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.498086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb5b19-7ac7-4ab1-8ebe-cde38ed88144 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:05.499962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859dd4c-dcc7-411a-bae4-ac5f89cce0fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a859dd4c-dcc7-411a-bae4-ac5f89cce0fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:05.507852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:05.508866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:05.509686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:05.513583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 236/489] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:05.515281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859dd4c-dcc7-411a-bae4-ac5f89cce0fb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:05.541259 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7a535-51b4-4cf5-aa08-788bfa17856b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:05.541996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7a535-51b4-4cf5-aa08-788bfa17856b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.542279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7a535-51b4-4cf5-aa08-788bfa17856b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.542545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7a535-51b4-4cf5-aa08-788bfa17856b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.549923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7a535-51b4-4cf5-aa08-788bfa17856b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:05.551379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7a535-51b4-4cf5-aa08-788bfa17856b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-c2d7a535-51b4-4cf5-aa08-788bfa17856b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:05.585602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:05.587845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:05.589991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:05.589991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:05.589991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:05.589991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:05.589991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:05.589991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:05.589991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:05.589991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:05.589991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:05.589991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:05.601894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 254/490] 10.222.0.50 () {56 vars in 1093 bytes} [Fri Apr 29 18:30:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:05.601894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7a535-51b4-4cf5-aa08-788bfa17856b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:05.628517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859dd4c-dcc7-411a-bae4-ac5f89cce0fb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:05.660410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdad71b-7c92-4577-b666-83cfaeb6c9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:05.661148 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad71b-7c92-4577-b666-83cfaeb6c9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.661448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad71b-7c92-4577-b666-83cfaeb6c9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.661718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad71b-7c92-4577-b666-83cfaeb6c9e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.662470 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad71b-7c92-4577-b666-83cfaeb6c9e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:05.662776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad71b-7c92-4577-b666-83cfaeb6c9e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:05.664149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad71b-7c92-4577-b666-83cfaeb6c9e7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:05.677185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 237/491] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:05.678669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fa09f-1916-4b20-8ede-00b7ce294242 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.678964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fa09f-1916-4b20-8ede-00b7ce294242 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.679251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fa09f-1916-4b20-8ede-00b7ce294242 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.681142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 238/492] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:05.682119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fa09f-1916-4b20-8ede-00b7ce294242 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:05.704908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7a535-51b4-4cf5-aa08-788bfa17856b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:05.741238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7f623-7385-4884-8226-a6571c21cbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:05.741917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f623-7385-4884-8226-a6571c21cbde None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.742192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f623-7385-4884-8226-a6571c21cbde None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.742458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f623-7385-4884-8226-a6571c21cbde None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.743375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7f623-7385-4884-8226-a6571c21cbde None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:05.743683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7f623-7385-4884-8226-a6571c21cbde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:05.745187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7f623-7385-4884-8226-a6571c21cbde None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:05.786970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 255/493] 10.222.0.50 () {56 vars in 1133 bytes} [Fri Apr 29 18:30:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:05.788592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ae43-47fa-4726-94f2-d6cc18b50a73 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.788699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fa09f-1916-4b20-8ede-00b7ce294242 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:05.788888 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ae43-47fa-4726-94f2-d6cc18b50a73 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.789156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ae43-47fa-4726-94f2-d6cc18b50a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.805666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5fe5ae43-47fa-4726-94f2-d6cc18b50a73 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:05.806899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5fe5ae43-47fa-4726-94f2-d6cc18b50a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:05.823033 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a94b0b-5e15-4838-977d-45fe56deb089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:05.823897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a94b0b-5e15-4838-977d-45fe56deb089 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.824183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a94b0b-5e15-4838-977d-45fe56deb089 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.824455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a94b0b-5e15-4838-977d-45fe56deb089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:05.835606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a94b0b-5e15-4838-977d-45fe56deb089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:05.837246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a94b0b-5e15-4838-977d-45fe56deb089 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c4a94b0b-5e15-4838-977d-45fe56deb089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:05.855335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:05.857368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:05.859355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:05.859355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:05.859355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:05.859355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:05.864527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 239/494] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:05.864527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a94b0b-5e15-4838-977d-45fe56deb089 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:05.874742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 256/495] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:05.875673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ae43-47fa-4726-94f2-d6cc18b50a73 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:05.962182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a94b0b-5e15-4838-977d-45fe56deb089 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:05.972040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ae43-47fa-4726-94f2-d6cc18b50a73 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:05.998114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b267521-6f53-418c-bec6-c0f9224ee3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:05.998768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b267521-6f53-418c-bec6-c0f9224ee3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:05.999053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b267521-6f53-418c-bec6-c0f9224ee3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:05.999329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b267521-6f53-418c-bec6-c0f9224ee3b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.004234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77ec3c-ab07-4a66-a8cf-929f50df4126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:06.004990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ec3c-ab07-4a66-a8cf-929f50df4126 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.005277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ec3c-ab07-4a66-a8cf-929f50df4126 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.005548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ec3c-ab07-4a66-a8cf-929f50df4126 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.006270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77ec3c-ab07-4a66-a8cf-929f50df4126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:06.006578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77ec3c-ab07-4a66-a8cf-929f50df4126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:06.008030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77ec3c-ab07-4a66-a8cf-929f50df4126 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:06.027727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 257/496] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:06.027727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96de5a-0401-4186-8b43-f231fd082d16 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.027727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96de5a-0401-4186-8b43-f231fd082d16 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.027727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96de5a-0401-4186-8b43-f231fd082d16 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.027727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 258/497] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:06.027727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96de5a-0401-4186-8b43-f231fd082d16 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.072759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b267521-6f53-418c-bec6-c0f9224ee3b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:06.074610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b267521-6f53-418c-bec6-c0f9224ee3b6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:06.121949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96de5a-0401-4186-8b43-f231fd082d16 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:06.154003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5db2a4-6019-4eb7-b757-3200e71eeb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:06.154828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5db2a4-6019-4eb7-b757-3200e71eeb87 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.155111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5db2a4-6019-4eb7-b757-3200e71eeb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.155382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5db2a4-6019-4eb7-b757-3200e71eeb87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.165908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5db2a4-6019-4eb7-b757-3200e71eeb87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-bf5db2a4-6019-4eb7-b757-3200e71eeb87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:06.173628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:06.174529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:06.180647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:06.181533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 240/499] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:06.181607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 259/499] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:06.181607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1b49-e85b-40ef-98c8-925fb1fa5604 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.181607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1b49-e85b-40ef-98c8-925fb1fa5604 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.181607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca1b49-e85b-40ef-98c8-925fb1fa5604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.181834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b267521-6f53-418c-bec6-c0f9224ee3b6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.197848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b9ca1b49-e85b-40ef-98c8-925fb1fa5604 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:06.199047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b9ca1b49-e85b-40ef-98c8-925fb1fa5604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:06.264856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 260/500] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:06.265699 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca1b49-e85b-40ef-98c8-925fb1fa5604 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.294145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b267521-6f53-418c-bec6-c0f9224ee3b6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:06.335264 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17470a8e-bfd9-47b7-a305-cbaee97acabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:06.336009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17470a8e-bfd9-47b7-a305-cbaee97acabf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.336345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17470a8e-bfd9-47b7-a305-cbaee97acabf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.336653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17470a8e-bfd9-47b7-a305-cbaee97acabf None admin] PATH_INFO: `/v3/users/glance` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.347754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17470a8e-bfd9-47b7-a305-cbaee97acabf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:06.349776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17470a8e-bfd9-47b7-a305-cbaee97acabf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:06.364796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca1b49-e85b-40ef-98c8-925fb1fa5604 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-17470a8e-bfd9-47b7-a305-cbaee97acabf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:06.371326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:06.372265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 241/501] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:06.378556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17470a8e-bfd9-47b7-a305-cbaee97acabf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.398596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd3bcf-aeac-44a3-b345-c45350ac3741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:06.399424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd3bcf-aeac-44a3-b345-c45350ac3741 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.399713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd3bcf-aeac-44a3-b345-c45350ac3741 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.400033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd3bcf-aeac-44a3-b345-c45350ac3741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.410602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd3bcf-aeac-44a3-b345-c45350ac3741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:06.412077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd3bcf-aeac-44a3-b345-c45350ac3741 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d0fd3bcf-aeac-44a3-b345-c45350ac3741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:06.434095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:06.435018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:06.435863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:06.435863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:06.435863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:06.435863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:06.440110 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 261/502] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:06.440643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd3bcf-aeac-44a3-b345-c45350ac3741 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.490556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17470a8e-bfd9-47b7-a305-cbaee97acabf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:06.528014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c528ef-aa97-4b50-84e6-c7e14a962c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:06.528702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c528ef-aa97-4b50-84e6-c7e14a962c1d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.528978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c528ef-aa97-4b50-84e6-c7e14a962c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.529254 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c528ef-aa97-4b50-84e6-c7e14a962c1d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.529982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c528ef-aa97-4b50-84e6-c7e14a962c1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:06.530283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c528ef-aa97-4b50-84e6-c7e14a962c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:06.531678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c528ef-aa97-4b50-84e6-c7e14a962c1d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:06.539410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd3bcf-aeac-44a3-b345-c45350ac3741 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:06.539873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 242/503] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:06.540635 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c528ef-aa97-4b50-84e6-c7e14a962c1d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.571480 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7efbe3-b4ac-4bbd-a840-6b4f26b94733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:06.572289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7efbe3-b4ac-4bbd-a840-6b4f26b94733 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.572601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7efbe3-b4ac-4bbd-a840-6b4f26b94733 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.572878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7efbe3-b4ac-4bbd-a840-6b4f26b94733 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.582430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7efbe3-b4ac-4bbd-a840-6b4f26b94733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-3d7efbe3-b4ac-4bbd-a840-6b4f26b94733 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:06.590299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:06.591282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:06.592072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 262/504] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:06.593139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7efbe3-b4ac-4bbd-a840-6b4f26b94733 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.636166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c528ef-aa97-4b50-84e6-c7e14a962c1d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:06.669273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505bbf6-97d2-493c-b742-2a5044492b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:06.669949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505bbf6-97d2-493c-b742-2a5044492b87 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.670231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505bbf6-97d2-493c-b742-2a5044492b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.670501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505bbf6-97d2-493c-b742-2a5044492b87 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.671326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505bbf6-97d2-493c-b742-2a5044492b87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:06.671626 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505bbf6-97d2-493c-b742-2a5044492b87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:06.673083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505bbf6-97d2-493c-b742-2a5044492b87 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:06.686576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7efbe3-b4ac-4bbd-a840-6b4f26b94733 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:06.708907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 243/505] 10.222.0.50 () {56 vars in 1137 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:06.710167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108c50-0383-4bbb-be5e-337a59907d0d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.710448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108c50-0383-4bbb-be5e-337a59907d0d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.710717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68108c50-0383-4bbb-be5e-337a59907d0d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.711938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 244/506] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:06.713293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68108c50-0383-4bbb-be5e-337a59907d0d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.720701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bd075-3ee9-4c62-97d2-e31b16a855aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:06.721366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bd075-3ee9-4c62-97d2-e31b16a855aa None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.721639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bd075-3ee9-4c62-97d2-e31b16a855aa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.721902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bd075-3ee9-4c62-97d2-e31b16a855aa None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.722614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bd075-3ee9-4c62-97d2-e31b16a855aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:06.722912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bd075-3ee9-4c62-97d2-e31b16a855aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:06.724284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bd075-3ee9-4c62-97d2-e31b16a855aa None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:06.735820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 263/507] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:06.736662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bd075-3ee9-4c62-97d2-e31b16a855aa None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.817129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68108c50-0383-4bbb-be5e-337a59907d0d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:06.834330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bd075-3ee9-4c62-97d2-e31b16a855aa None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:06.852865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ff75e-7fd7-454d-93e0-dbbd98648ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:06.853598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ff75e-7fd7-454d-93e0-dbbd98648ced None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.853873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ff75e-7fd7-454d-93e0-dbbd98648ced None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.854138 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ff75e-7fd7-454d-93e0-dbbd98648ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.863706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ff75e-7fd7-454d-93e0-dbbd98648ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:06.865202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ff75e-7fd7-454d-93e0-dbbd98648ced None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:06.869517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939130e-1001-4d42-933f-349c4af043a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:06.870174 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939130e-1001-4d42-933f-349c4af043a7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.870455 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939130e-1001-4d42-933f-349c4af043a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.870722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939130e-1001-4d42-933f-349c4af043a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.871444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939130e-1001-4d42-933f-349c4af043a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:06.871743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939130e-1001-4d42-933f-349c4af043a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:06.873133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939130e-1001-4d42-933f-349c4af043a7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:06.880736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 264/508] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b10ff75e-7fd7-454d-93e0-dbbd98648ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:06.882260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:06.883102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:06.884075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:06.884075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:06.884075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:06.884075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:06.884722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939130e-1001-4d42-933f-349c4af043a7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.884810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 245/509] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:06.884810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e4723-2935-43a2-a585-897a737d9277 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:06.884942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e4723-2935-43a2-a585-897a737d9277 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:06.885541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e4723-2935-43a2-a585-897a737d9277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:06.901836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-db2e4723-2935-43a2-a585-897a737d9277 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:06.903012 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-db2e4723-2935-43a2-a585-897a737d9277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:06.966754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 246/510] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:06.967940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e4723-2935-43a2-a585-897a737d9277 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:06.998796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939130e-1001-4d42-933f-349c4af043a7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.032972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe66f3c-b6b2-48e4-8f05-7544b03f4019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.033736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe66f3c-b6b2-48e4-8f05-7544b03f4019 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.034010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe66f3c-b6b2-48e4-8f05-7544b03f4019 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.034301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe66f3c-b6b2-48e4-8f05-7544b03f4019 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.060682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e4723-2935-43a2-a585-897a737d9277 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.098380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe66f3c-b6b2-48e4-8f05-7544b03f4019 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.099849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe66f3c-b6b2-48e4-8f05-7544b03f4019 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.100804 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7f45e-ddd8-4bef-a6f0-534f3c732d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.101597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7f45e-ddd8-4bef-a6f0-534f3c732d57 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.101881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7f45e-ddd8-4bef-a6f0-534f3c732d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.102156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7f45e-ddd8-4bef-a6f0-534f3c732d57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.109062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7f45e-ddd8-4bef-a6f0-534f3c732d57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.110451 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7f45e-ddd8-4bef-a6f0-534f3c732d57 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-22d7f45e-ddd8-4bef-a6f0-534f3c732d57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:07.129662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:07.130623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:07.131468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:07.131468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:07.131468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:07.131468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:07.131468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:07.131468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:07.131468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:07.131468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:07.131468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 247/511] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:07.131933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7f45e-ddd8-4bef-a6f0-534f3c732d57 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.204838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 265/512] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:07.205685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe66f3c-b6b2-48e4-8f05-7544b03f4019 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.233097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7f45e-ddd8-4bef-a6f0-534f3c732d57 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.266064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53319d75-27d4-4bdd-b377-a334d419a912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.266858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53319d75-27d4-4bdd-b377-a334d419a912 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.267139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53319d75-27d4-4bdd-b377-a334d419a912 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.267403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53319d75-27d4-4bdd-b377-a334d419a912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.277661 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53319d75-27d4-4bdd-b377-a334d419a912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.279118 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53319d75-27d4-4bdd-b377-a334d419a912 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-53319d75-27d4-4bdd-b377-a334d419a912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:07.296471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:07.297358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:07.298583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:07.298583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:07.298583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:07.298583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:07.298583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 248/513] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:07.298583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53319d75-27d4-4bdd-b377-a334d419a912 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.304268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe66f3c-b6b2-48e4-8f05-7544b03f4019 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.337624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845437a0-7723-410d-b6e4-0dfb3ee062cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.338299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845437a0-7723-410d-b6e4-0dfb3ee062cd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.338569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845437a0-7723-410d-b6e4-0dfb3ee062cd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.338794 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845437a0-7723-410d-b6e4-0dfb3ee062cd None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.339526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845437a0-7723-410d-b6e4-0dfb3ee062cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:07.339824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845437a0-7723-410d-b6e4-0dfb3ee062cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.341197 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845437a0-7723-410d-b6e4-0dfb3ee062cd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.352359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 266/514] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:07.353043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845437a0-7723-410d-b6e4-0dfb3ee062cd None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.397705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53319d75-27d4-4bdd-b377-a334d419a912 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.429239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd84be7-3ba2-4700-b502-7e1e01346ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.430030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd84be7-3ba2-4700-b502-7e1e01346ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.430300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd84be7-3ba2-4700-b502-7e1e01346ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.430572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd84be7-3ba2-4700-b502-7e1e01346ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.440759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd84be7-3ba2-4700-b502-7e1e01346ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.442207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd84be7-3ba2-4700-b502-7e1e01346ec4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.444083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845437a0-7723-410d-b6e4-0dfb3ee062cd None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-cfd84be7-3ba2-4700-b502-7e1e01346ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:07.458585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:07.459588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:07.460442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:07.460442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:07.460442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:07.460442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:07.460442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 249/515] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:07.460751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd84be7-3ba2-4700-b502-7e1e01346ec4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.477800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9394fa2-4356-4242-9f8a-7b84a43cd75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.478909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9394fa2-4356-4242-9f8a-7b84a43cd75f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.479190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9394fa2-4356-4242-9f8a-7b84a43cd75f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.479464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9394fa2-4356-4242-9f8a-7b84a43cd75f None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.480244 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9394fa2-4356-4242-9f8a-7b84a43cd75f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:07.480569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9394fa2-4356-4242-9f8a-7b84a43cd75f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.481998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9394fa2-4356-4242-9f8a-7b84a43cd75f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.499409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 267/516] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:07.500189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9394fa2-4356-4242-9f8a-7b84a43cd75f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.553965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd84be7-3ba2-4700-b502-7e1e01346ec4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.587555 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa34a01-6493-4ac6-a921-6d9cac6ec153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.588370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa34a01-6493-4ac6-a921-6d9cac6ec153 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.588674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa34a01-6493-4ac6-a921-6d9cac6ec153 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.588948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa34a01-6493-4ac6-a921-6d9cac6ec153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.596929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9394fa2-4356-4242-9f8a-7b84a43cd75f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.601811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa34a01-6493-4ac6-a921-6d9cac6ec153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.601811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa34a01-6493-4ac6-a921-6d9cac6ec153 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8aa34a01-6493-4ac6-a921-6d9cac6ec153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:07.623311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:07.624208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:07.625098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:07.625098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:07.625098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:07.625098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:07.625098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 250/517] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:07.625098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa34a01-6493-4ac6-a921-6d9cac6ec153 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.632880 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f6e7e-9777-43ba-b252-bd3a81839026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.633554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f6e7e-9777-43ba-b252-bd3a81839026 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.633891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f6e7e-9777-43ba-b252-bd3a81839026 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.634299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f6e7e-9777-43ba-b252-bd3a81839026 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.635027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f6e7e-9777-43ba-b252-bd3a81839026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:07.635330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f6e7e-9777-43ba-b252-bd3a81839026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.636749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f6e7e-9777-43ba-b252-bd3a81839026 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.648025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 268/518] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:07.648775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f6e7e-9777-43ba-b252-bd3a81839026 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.718437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa34a01-6493-4ac6-a921-6d9cac6ec153 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.743740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f6e7e-9777-43ba-b252-bd3a81839026 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.750864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a481acb-0cc8-42fe-8968-75d0ad3439b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.751645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a481acb-0cc8-42fe-8968-75d0ad3439b4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.751918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a481acb-0cc8-42fe-8968-75d0ad3439b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.752225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a481acb-0cc8-42fe-8968-75d0ad3439b4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.760376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a481acb-0cc8-42fe-8968-75d0ad3439b4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.761862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a481acb-0cc8-42fe-8968-75d0ad3439b4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.774941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5db0a5-0feb-4b7a-8abf-b633c83b5614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1a481acb-0cc8-42fe-8968-75d0ad3439b4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:07.775536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:07.776435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:07.777639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:07.777639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:07.777639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:07.777639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:07.777639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:07.777639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:07.777639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:07.777639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:07.777639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Apr 29 18:30:07.777639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:07.778382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5db0a5-0feb-4b7a-8abf-b633c83b5614 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.778382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5db0a5-0feb-4b7a-8abf-b633c83b5614 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.778382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5db0a5-0feb-4b7a-8abf-b633c83b5614 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.778382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5db0a5-0feb-4b7a-8abf-b633c83b5614 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:07.778618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 251/519] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:07.778681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5db0a5-0feb-4b7a-8abf-b633c83b5614 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.778743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a481acb-0cc8-42fe-8968-75d0ad3439b4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.778814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5db0a5-0feb-4b7a-8abf-b633c83b5614 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.789796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 269/520] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:07.790613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5db0a5-0feb-4b7a-8abf-b633c83b5614 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.879330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a481acb-0cc8-42fe-8968-75d0ad3439b4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.882099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5db0a5-0feb-4b7a-8abf-b633c83b5614 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:07.914057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140881c-91f8-45d5-b1c3-833e18a0c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.914257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d66d6-ceb8-44f4-a76e-43747225b33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:07.914640 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d66d6-ceb8-44f4-a76e-43747225b33c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.914728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140881c-91f8-45d5-b1c3-833e18a0c231 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:07.915160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d66d6-ceb8-44f4-a76e-43747225b33c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.915274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140881c-91f8-45d5-b1c3-833e18a0c231 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:07.915356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d66d6-ceb8-44f4-a76e-43747225b33c None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.915444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140881c-91f8-45d5-b1c3-833e18a0c231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:07.916017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d66d6-ceb8-44f4-a76e-43747225b33c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:07.916184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140881c-91f8-45d5-b1c3-833e18a0c231 None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=341358254d884b278d7fa19d95c2dc2d) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:07.916327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d66d6-ceb8-44f4-a76e-43747225b33c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.916469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140881c-91f8-45d5-b1c3-833e18a0c231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:07.917698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d66d6-ceb8-44f4-a76e-43747225b33c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.917959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140881c-91f8-45d5-b1c3-833e18a0c231 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:07.923527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 252/521] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/role_assignments?user.id=341358254d884b278d7fa19d95c2dc2d&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:07.924294 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140881c-91f8-45d5-b1c3-833e18a0c231 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:07.929147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 270/522] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:07.930319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d66d6-ceb8-44f4-a76e-43747225b33c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:08.029492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140881c-91f8-45d5-b1c3-833e18a0c231 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:08.041404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d66d6-ceb8-44f4-a76e-43747225b33c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:08.063332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5479e-71c2-4e90-b52e-c6bee95301da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:08.064149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5479e-71c2-4e90-b52e-c6bee95301da None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:08.064426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5479e-71c2-4e90-b52e-c6bee95301da None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:08.064724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5479e-71c2-4e90-b52e-c6bee95301da None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:08.073284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5479e-71c2-4e90-b52e-c6bee95301da None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:08.074727 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5479e-71c2-4e90-b52e-c6bee95301da None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:08.075631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27719000-855a-4034-acf0-cccf1531b077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:08.076315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27719000-855a-4034-acf0-cccf1531b077 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:08.076622 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27719000-855a-4034-acf0-cccf1531b077 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:08.076895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27719000-855a-4034-acf0-cccf1531b077 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:08.077739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27719000-855a-4034-acf0-cccf1531b077 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:08.078041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27719000-855a-4034-acf0-cccf1531b077 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:08.079477 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27719000-855a-4034-acf0-cccf1531b077 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b1c5479e-71c2-4e90-b52e-c6bee95301da None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:08.089785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:08.090880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:08.092003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:08.092003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:08.092003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:08.092003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:08.092003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:08.092003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:08.092003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:08.092003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:08.092003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Apr 29 18:30:08.092003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:08.096214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 253/523] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:08.097177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5479e-71c2-4e90-b52e-c6bee95301da None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:08.101501 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 271/524] 10.222.0.50 () {56 vars in 1137 bytes} [Fri Apr 29 18:30:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:08.104527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27719000-855a-4034-acf0-cccf1531b077 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:08.195502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5479e-71c2-4e90-b52e-c6bee95301da None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:08.203398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27719000-855a-4034-acf0-cccf1531b077 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:08.227973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835f1b-0047-40db-9f3f-f84963491491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:08.228672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835f1b-0047-40db-9f3f-f84963491491 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:08.228939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835f1b-0047-40db-9f3f-f84963491491 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:08.229211 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835f1b-0047-40db-9f3f-f84963491491 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:08.229916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18835f1b-0047-40db-9f3f-f84963491491 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:08.231364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18835f1b-0047-40db-9f3f-f84963491491 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:08.237535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a3ef5-e79b-4be1-ba94-094608928817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:08.238336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a3ef5-e79b-4be1-ba94-094608928817 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:08.238607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a3ef5-e79b-4be1-ba94-094608928817 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:08.238863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a3ef5-e79b-4be1-ba94-094608928817 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:08.246243 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a3ef5-e79b-4be1-ba94-094608928817 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:08.247818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a3ef5-e79b-4be1-ba94-094608928817 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:08.260742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 254/525] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:30:08 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:08.261776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835f1b-0047-40db-9f3f-f84963491491 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-713a3ef5-e79b-4be1-ba94-094608928817 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:08.262111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:08.263102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:08.263934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:08.263934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:08.263934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:08.263934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:08.263934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:08.263934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:08.263934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:08.263934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:08.268435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 272/526] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:08.269503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a3ef5-e79b-4be1-ba94-094608928817 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:08.375275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a3ef5-e79b-4be1-ba94-094608928817 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:08.375871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835f1b-0047-40db-9f3f-f84963491491 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:08.411242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc80653-59d9-4bf7-a3cd-23d69ad6df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:08.411935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc80653-59d9-4bf7-a3cd-23d69ad6df46 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:08.412181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251353b9-f890-44e6-bd17-91ed4d77c652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:08.412291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc80653-59d9-4bf7-a3cd-23d69ad6df46 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:08.412458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc80653-59d9-4bf7-a3cd-23d69ad6df46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:08.412895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251353b9-f890-44e6-bd17-91ed4d77c652 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:08.413292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251353b9-f890-44e6-bd17-91ed4d77c652 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:08.413572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251353b9-f890-44e6-bd17-91ed4d77c652 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:08.414449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251353b9-f890-44e6-bd17-91ed4d77c652 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:08.414758 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251353b9-f890-44e6-bd17-91ed4d77c652 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:08.416300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251353b9-f890-44e6-bd17-91ed4d77c652 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:08.438278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 255/527] 10.222.0.50 () {56 vars in 1139 bytes} [Fri Apr 29 18:30:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:08.439514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251353b9-f890-44e6-bd17-91ed4d77c652 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:08.487644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc80653-59d9-4bf7-a3cd-23d69ad6df46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:08.489451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc80653-59d9-4bf7-a3cd-23d69ad6df46 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:08.542187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251353b9-f890-44e6-bd17-91ed4d77c652 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:08.576532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82dd93-1945-4bde-baf9-d658a7061a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:08.577311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82dd93-1945-4bde-baf9-d658a7061a61 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:08.577468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82dd93-1945-4bde-baf9-d658a7061a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:08.577748 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82dd93-1945-4bde-baf9-d658a7061a61 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:08.578560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82dd93-1945-4bde-baf9-d658a7061a61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:08.578862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82dd93-1945-4bde-baf9-d658a7061a61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:08.580308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82dd93-1945-4bde-baf9-d658a7061a61 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:08.586506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 273/528] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:08.587365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc80653-59d9-4bf7-a3cd-23d69ad6df46 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:08.599188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 256/529] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:08.599891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82dd93-1945-4bde-baf9-d658a7061a61 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:08.687796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc80653-59d9-4bf7-a3cd-23d69ad6df46 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:08.697892 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82dd93-1945-4bde-baf9-d658a7061a61 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:08.721704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d373d6-7c4a-4dec-ad56-8cd40bc23a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:08.722362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d373d6-7c4a-4dec-ad56-8cd40bc23a0c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:08.722638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d373d6-7c4a-4dec-ad56-8cd40bc23a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:08.722951 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d373d6-7c4a-4dec-ad56-8cd40bc23a0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:08.732472 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd4c44-3459-495f-9146-8ee00420c9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:08.733318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd4c44-3459-495f-9146-8ee00420c9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:08.733600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd4c44-3459-495f-9146-8ee00420c9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:08.733876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd4c44-3459-495f-9146-8ee00420c9d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:08.744387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd4c44-3459-495f-9146-8ee00420c9d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:08.745735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd4c44-3459-495f-9146-8ee00420c9d0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3dbd4c44-3459-495f-9146-8ee00420c9d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:08.763126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:08.764022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:08.764894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:08.764894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:08.764894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:08.764894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:08.764894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 257/530] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:08.765577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd4c44-3459-495f-9146-8ee00420c9d0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:08.793689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d373d6-7c4a-4dec-ad56-8cd40bc23a0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:08.795604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d373d6-7c4a-4dec-ad56-8cd40bc23a0c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:08.882290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd4c44-3459-495f-9146-8ee00420c9d0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:08.908563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 274/531] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:08.909680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d373d6-7c4a-4dec-ad56-8cd40bc23a0c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:08.922308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ed682-9ecc-48a6-aba1-476252892522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:08.923275 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ed682-9ecc-48a6-aba1-476252892522 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:08.923585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ed682-9ecc-48a6-aba1-476252892522 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:08.924082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ed682-9ecc-48a6-aba1-476252892522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:08.925130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ed682-9ecc-48a6-aba1-476252892522 None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=6552c41f1c654ef5bfdd940e3b807dae) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:08.925663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ed682-9ecc-48a6-aba1-476252892522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:08.927766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ed682-9ecc-48a6-aba1-476252892522 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:08.934895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 258/532] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:08 2022] GET /identity/v3/role_assignments?user.id=6552c41f1c654ef5bfdd940e3b807dae&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:08.935926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ed682-9ecc-48a6-aba1-476252892522 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:09.075741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d373d6-7c4a-4dec-ad56-8cd40bc23a0c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:09.122272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c74461-6fb5-44b6-be3c-f526f0aa8bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:09.122963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c74461-6fb5-44b6-be3c-f526f0aa8bda None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:09.123238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c74461-6fb5-44b6-be3c-f526f0aa8bda None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:09.123507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c74461-6fb5-44b6-be3c-f526f0aa8bda None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:09.124230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c74461-6fb5-44b6-be3c-f526f0aa8bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:09.124550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c74461-6fb5-44b6-be3c-f526f0aa8bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:09.128608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c74461-6fb5-44b6-be3c-f526f0aa8bda None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:09.142091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 275/533] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:09.147488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c74461-6fb5-44b6-be3c-f526f0aa8bda None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:09.162192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ed682-9ecc-48a6-aba1-476252892522 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:09.196206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11246420-e6af-4c16-83f2-984cba4a4746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:09.197382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11246420-e6af-4c16-83f2-984cba4a4746 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:09.197876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11246420-e6af-4c16-83f2-984cba4a4746 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:09.198331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11246420-e6af-4c16-83f2-984cba4a4746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:09.209556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11246420-e6af-4c16-83f2-984cba4a4746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:09.211295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11246420-e6af-4c16-83f2-984cba4a4746 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-11246420-e6af-4c16-83f2-984cba4a4746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:09.230086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:09.231190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:09.232375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:09.232375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:09.232375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:09.232375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:09.232375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 259/534] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:09.238287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11246420-e6af-4c16-83f2-984cba4a4746 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:09.262236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c74461-6fb5-44b6-be3c-f526f0aa8bda None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:09.326458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310a444-91d3-4a28-9012-1e8a80169e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:09.327188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310a444-91d3-4a28-9012-1e8a80169e46 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:09.327461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310a444-91d3-4a28-9012-1e8a80169e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:09.327727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310a444-91d3-4a28-9012-1e8a80169e46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:09.339384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310a444-91d3-4a28-9012-1e8a80169e46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:09.340883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310a444-91d3-4a28-9012-1e8a80169e46 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-9310a444-91d3-4a28-9012-1e8a80169e46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:09.363082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:09.363980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:09.364873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:09.364873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:09.364873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:09.364873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:09.364873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:09.364873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:09.364873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:09.364873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:09.368716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 276/535] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:09.370302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310a444-91d3-4a28-9012-1e8a80169e46 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:09.432423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11246420-e6af-4c16-83f2-984cba4a4746 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:09.466754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310a444-91d3-4a28-9012-1e8a80169e46 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:09.501625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7faa2-b274-43a8-aba3-f5606fd0ccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:09.502296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7faa2-b274-43a8-aba3-f5606fd0ccca None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:09.502571 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7faa2-b274-43a8-aba3-f5606fd0ccca None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:09.502841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7faa2-b274-43a8-aba3-f5606fd0ccca None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:09.503623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7faa2-b274-43a8-aba3-f5606fd0ccca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:09.503953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7faa2-b274-43a8-aba3-f5606fd0ccca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:09.505390 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7faa2-b274-43a8-aba3-f5606fd0ccca None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:09.522643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 277/536] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:09.523953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7faa2-b274-43a8-aba3-f5606fd0ccca None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:09.542297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e77cf7-f32d-4416-a4e1-416d91be76e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:09.543286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e77cf7-f32d-4416-a4e1-416d91be76e3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:09.543706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e77cf7-f32d-4416-a4e1-416d91be76e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:09.544109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e77cf7-f32d-4416-a4e1-416d91be76e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:09.545354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e77cf7-f32d-4416-a4e1-416d91be76e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:09.545701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e77cf7-f32d-4416-a4e1-416d91be76e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:09.547105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e77cf7-f32d-4416-a4e1-416d91be76e3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:09.583729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 260/537] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:09.589669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e77cf7-f32d-4416-a4e1-416d91be76e3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:09.627873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7faa2-b274-43a8-aba3-f5606fd0ccca None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:09.660805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d5c86-3d94-44a4-8ae6-27c7ba1b7d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:09.661468 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d5c86-3d94-44a4-8ae6-27c7ba1b7d49 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:09.661746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d5c86-3d94-44a4-8ae6-27c7ba1b7d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:09.662016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d5c86-3d94-44a4-8ae6-27c7ba1b7d49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:09.662833 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d5c86-3d94-44a4-8ae6-27c7ba1b7d49 None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=ae4fe640bb564a8abe872cb1ec74dc7d) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:09.663137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d5c86-3d94-44a4-8ae6-27c7ba1b7d49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:09.664578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d5c86-3d94-44a4-8ae6-27c7ba1b7d49 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:09.676635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 278/538] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:09 2022] GET /identity/v3/role_assignments?user.id=ae4fe640bb564a8abe872cb1ec74dc7d&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:09.812210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e77cf7-f32d-4416-a4e1-416d91be76e3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:09.866235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d85e0-5640-4fbf-873e-6bcf0241cddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:09.866985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d85e0-5640-4fbf-873e-6bcf0241cddc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:09.867265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d85e0-5640-4fbf-873e-6bcf0241cddc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:09.867530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d85e0-5640-4fbf-873e-6bcf0241cddc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:09.875735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d85e0-5640-4fbf-873e-6bcf0241cddc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:09.877232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d85e0-5640-4fbf-873e-6bcf0241cddc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d41d85e0-5640-4fbf-873e-6bcf0241cddc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:09.891463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:09.892388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:09.893271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:09.893271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:09.893271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:09.893271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:09.893271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:09.893271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:09.893271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:09.893271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:09.893271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 261/539] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:09.901675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d5c86-3d94-44a4-8ae6-27c7ba1b7d49 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:10.034319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d5c86-3d94-44a4-8ae6-27c7ba1b7d49 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:10.079970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366308e-4fd6-44f7-868f-38a37a4cb5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:10.080966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366308e-4fd6-44f7-868f-38a37a4cb5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:10.081380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366308e-4fd6-44f7-868f-38a37a4cb5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:10.081782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366308e-4fd6-44f7-868f-38a37a4cb5b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:10.082700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366308e-4fd6-44f7-868f-38a37a4cb5b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:10.083145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366308e-4fd6-44f7-868f-38a37a4cb5b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:10.084742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366308e-4fd6-44f7-868f-38a37a4cb5b6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:10.106281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 279/540] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:10.112531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d85e0-5640-4fbf-873e-6bcf0241cddc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:10.242341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d85e0-5640-4fbf-873e-6bcf0241cddc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:10.285230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84391e-c346-4073-aafc-9f6351716495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:10.285894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84391e-c346-4073-aafc-9f6351716495 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:10.286185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84391e-c346-4073-aafc-9f6351716495 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:10.286449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84391e-c346-4073-aafc-9f6351716495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:10.287278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84391e-c346-4073-aafc-9f6351716495 None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=07849b2925334763b119111ed52dd34c) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:10.287587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84391e-c346-4073-aafc-9f6351716495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:10.289056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84391e-c346-4073-aafc-9f6351716495 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:10.295263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 262/541] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:10 2022] GET /identity/v3/role_assignments?user.id=07849b2925334763b119111ed52dd34c&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:10.540729 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbd383-69a9-45a2-9dad-3fa3320d3560 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:10.541070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbd383-69a9-45a2-9dad-3fa3320d3560 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:10.541179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbd383-69a9-45a2-9dad-3fa3320d3560 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:10.542789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 280/542] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:10.552136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bc160-8b07-4006-9569-50245017a5bc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:10.552415 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bc160-8b07-4006-9569-50245017a5bc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:10.552720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bc160-8b07-4006-9569-50245017a5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:10.594167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a74bc160-8b07-4006-9569-50245017a5bc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:10.595396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a74bc160-8b07-4006-9569-50245017a5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:10.726079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 263/543] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:30:10.757629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbd383-69a9-45a2-9dad-3fa3320d3560 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:10.886195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbd383-69a9-45a2-9dad-3fa3320d3560 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:10.938352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee53d55-ff14-48b3-8db4-da5d1e483564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:10.939214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee53d55-ff14-48b3-8db4-da5d1e483564 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:10.939496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee53d55-ff14-48b3-8db4-da5d1e483564 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:10.939765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee53d55-ff14-48b3-8db4-da5d1e483564 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:10.950403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee53d55-ff14-48b3-8db4-da5d1e483564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-9ee53d55-ff14-48b3-8db4-da5d1e483564 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:10.966563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:10.967892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:10.968762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 281/544] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:10.976594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bc160-8b07-4006-9569-50245017a5bc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:11.035451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825b1a7-8a92-4a6a-bc1c-c9ac98d6b8cd None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:11.035819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825b1a7-8a92-4a6a-bc1c-c9ac98d6b8cd None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:11.035999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825b1a7-8a92-4a6a-bc1c-c9ac98d6b8cd None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:11.037712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 282/545] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:11.044573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fa37e-c4e7-4c55-be23-cf4f99b29739 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:11.044573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fa37e-c4e7-4c55-be23-cf4f99b29739 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:11.044573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fa37e-c4e7-4c55-be23-cf4f99b29739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:11.063374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-005fa37e-c4e7-4c55-be23-cf4f99b29739 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:11.065167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-005fa37e-c4e7-4c55-be23-cf4f99b29739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:11.157428 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bc160-8b07-4006-9569-50245017a5bc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:11.162871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 283/546] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:30:11.192729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbf1a3-0020-4499-8ed5-2155c29b2870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:11.193442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbf1a3-0020-4499-8ed5-2155c29b2870 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:11.193721 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbf1a3-0020-4499-8ed5-2155c29b2870 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:11.193987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbf1a3-0020-4499-8ed5-2155c29b2870 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:11.194725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbf1a3-0020-4499-8ed5-2155c29b2870 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:11.195021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbf1a3-0020-4499-8ed5-2155c29b2870 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:11.196436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbf1a3-0020-4499-8ed5-2155c29b2870 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:11.204961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 264/547] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:11.210618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fa37e-c4e7-4c55-be23-cf4f99b29739 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:11.221582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbf1a3-0020-4499-8ed5-2155c29b2870 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:11.352051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fa37e-c4e7-4c55-be23-cf4f99b29739 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:11.372349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbf1a3-0020-4499-8ed5-2155c29b2870 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:11.385000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd07ba-a2c9-4b7d-bae2-02fce4cdeaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:11.385799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd07ba-a2c9-4b7d-bae2-02fce4cdeaaa None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:11.386074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd07ba-a2c9-4b7d-bae2-02fce4cdeaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:11.386338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd07ba-a2c9-4b7d-bae2-02fce4cdeaaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:11.396479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd07ba-a2c9-4b7d-bae2-02fce4cdeaaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:11.397990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd07ba-a2c9-4b7d-bae2-02fce4cdeaaa None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:11.404721 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6378b-6155-452a-9477-518e72c0bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:11.405527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6378b-6155-452a-9477-518e72c0bb48 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:11.405804 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6378b-6155-452a-9477-518e72c0bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:11.406072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6378b-6155-452a-9477-518e72c0bb48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5cbd07ba-a2c9-4b7d-bae2-02fce4cdeaaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:11.415738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:11.416596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:11.417542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:11.417542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:11.417542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:11.417542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:11.417758 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6378b-6155-452a-9477-518e72c0bb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:11.417836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 284/548] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:11.422715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd07ba-a2c9-4b7d-bae2-02fce4cdeaaa None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-fea6378b-6155-452a-9477-518e72c0bb48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:11.424707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:11.426672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:11.428578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 265/549] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:11.434239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6378b-6155-452a-9477-518e72c0bb48 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:11.521700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd07ba-a2c9-4b7d-bae2-02fce4cdeaaa None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:11.555615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f710-fbb3-498a-a473-c1b8afe4d762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:11.556349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f710-fbb3-498a-a473-c1b8afe4d762 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:11.556840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f710-fbb3-498a-a473-c1b8afe4d762 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:11.557118 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f710-fbb3-498a-a473-c1b8afe4d762 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:11.557884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2f710-fbb3-498a-a473-c1b8afe4d762 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:11.558188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2f710-fbb3-498a-a473-c1b8afe4d762 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:11.559558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2f710-fbb3-498a-a473-c1b8afe4d762 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:11.567692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6378b-6155-452a-9477-518e72c0bb48 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:11.572123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 285/550] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:11.577650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f710-fbb3-498a-a473-c1b8afe4d762 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:11.609674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de471-a21a-4d92-b1b4-eb79c47284b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:11.610360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de471-a21a-4d92-b1b4-eb79c47284b2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:11.616514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de471-a21a-4d92-b1b4-eb79c47284b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:11.616514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de471-a21a-4d92-b1b4-eb79c47284b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:11.616514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473de471-a21a-4d92-b1b4-eb79c47284b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:11.616514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473de471-a21a-4d92-b1b4-eb79c47284b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:11.616514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473de471-a21a-4d92-b1b4-eb79c47284b2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:11.635614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 266/551] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:11.638625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de471-a21a-4d92-b1b4-eb79c47284b2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:11.753104 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f710-fbb3-498a-a473-c1b8afe4d762 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:11.803228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92d8d0-50e5-4e62-b2ae-dcf795db8f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:11.804444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92d8d0-50e5-4e62-b2ae-dcf795db8f84 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:11.804976 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92d8d0-50e5-4e62-b2ae-dcf795db8f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:11.805534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92d8d0-50e5-4e62-b2ae-dcf795db8f84 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:11.821989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92d8d0-50e5-4e62-b2ae-dcf795db8f84 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:11.823866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92d8d0-50e5-4e62-b2ae-dcf795db8f84 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:11.825400 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de471-a21a-4d92-b1b4-eb79c47284b2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-db92d8d0-50e5-4e62-b2ae-dcf795db8f84 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:11.861293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:11.862240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:11.863149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 286/552] 10.222.0.50 () {56 vars in 1093 bytes} [Fri Apr 29 18:30:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:11.869209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92d8d0-50e5-4e62-b2ae-dcf795db8f84 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:11.871860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf048d2-660e-451b-8e86-ae7f7046676c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:11.872696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf048d2-660e-451b-8e86-ae7f7046676c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:11.872980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf048d2-660e-451b-8e86-ae7f7046676c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:11.873252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf048d2-660e-451b-8e86-ae7f7046676c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:11.895506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf048d2-660e-451b-8e86-ae7f7046676c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:11.897014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf048d2-660e-451b-8e86-ae7f7046676c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ecf048d2-660e-451b-8e86-ae7f7046676c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:11.936951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:11.937868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:11.938698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:11.938698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:11.938698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:11.938698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:11.941003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 267/553] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:11.948096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf048d2-660e-451b-8e86-ae7f7046676c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:12.002965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92d8d0-50e5-4e62-b2ae-dcf795db8f84 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:12.044799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f41be0-e532-4af0-bbb4-a58c23a58f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:12.045401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f41be0-e532-4af0-bbb4-a58c23a58f6e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.045680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f41be0-e532-4af0-bbb4-a58c23a58f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.045950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f41be0-e532-4af0-bbb4-a58c23a58f6e None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.046785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f41be0-e532-4af0-bbb4-a58c23a58f6e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:12.047088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f41be0-e532-4af0-bbb4-a58c23a58f6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:12.048567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f41be0-e532-4af0-bbb4-a58c23a58f6e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:12.068968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 287/554] 10.222.0.50 () {56 vars in 1133 bytes} [Fri Apr 29 18:30:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:12.070626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfcc3f-1770-44fe-86bd-861849b170a9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.070918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfcc3f-1770-44fe-86bd-861849b170a9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.071673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfcc3f-1770-44fe-86bd-861849b170a9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.073691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 288/555] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:12.074392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfcc3f-1770-44fe-86bd-861849b170a9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:12.137873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf048d2-660e-451b-8e86-ae7f7046676c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:12.178366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfcc3f-1770-44fe-86bd-861849b170a9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:12.178891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bf623-6c07-4fd7-8373-de6632922044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:12.179567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf623-6c07-4fd7-8373-de6632922044 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.180012 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf623-6c07-4fd7-8373-de6632922044 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.180293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf623-6c07-4fd7-8373-de6632922044 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.182560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bf623-6c07-4fd7-8373-de6632922044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:12.182875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bf623-6c07-4fd7-8373-de6632922044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:12.184289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bf623-6c07-4fd7-8373-de6632922044 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:12.202962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 268/556] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:12.204142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a90bc-fe3b-455b-8234-ed4e9e6a9351 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.204433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a90bc-fe3b-455b-8234-ed4e9e6a9351 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.209351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a90bc-fe3b-455b-8234-ed4e9e6a9351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.224944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b650c-727e-4601-bcb3-ba415be8b1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:12.225810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b650c-727e-4601-bcb3-ba415be8b1dc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.226087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b650c-727e-4601-bcb3-ba415be8b1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.226359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b650c-727e-4601-bcb3-ba415be8b1dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.227154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-695a90bc-fe3b-455b-8234-ed4e9e6a9351 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:12.228788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-695a90bc-fe3b-455b-8234-ed4e9e6a9351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:12.240301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b650c-727e-4601-bcb3-ba415be8b1dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:12.242088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b650c-727e-4601-bcb3-ba415be8b1dc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-3c7b650c-727e-4601-bcb3-ba415be8b1dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:12.266604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:12.268417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:12.270137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:12.270137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:12.270137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:12.270137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:12.270137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 289/557] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:12.270137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b650c-727e-4601-bcb3-ba415be8b1dc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:12.304737 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 269/558] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:12.306519 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a90bc-fe3b-455b-8234-ed4e9e6a9351 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:12.400724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b650c-727e-4601-bcb3-ba415be8b1dc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:12.431180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a90bc-fe3b-455b-8234-ed4e9e6a9351 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:12.435353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe3ae-7e06-42d6-917d-139d1358f1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:12.436231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe3ae-7e06-42d6-917d-139d1358f1ae None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.436679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe3ae-7e06-42d6-917d-139d1358f1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.437019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe3ae-7e06-42d6-917d-139d1358f1ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.448516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbe3ae-7e06-42d6-917d-139d1358f1ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:12.448516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbe3ae-7e06-42d6-917d-139d1358f1ae None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-91bbe3ae-7e06-42d6-917d-139d1358f1ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:12.464519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:12.466422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:12.468799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:12.468799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:12.468799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:12.468799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:12.468799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:12.468799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:12.468799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:12.468799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:12.468799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:12.468799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:12.470063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f0fbf-e9e5-471c-9ca5-d23bb9335673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:12.470234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 290/559] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:12.470369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f0fbf-e9e5-471c-9ca5-d23bb9335673 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.470369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f0fbf-e9e5-471c-9ca5-d23bb9335673 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.470369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f0fbf-e9e5-471c-9ca5-d23bb9335673 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.476726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe3ae-7e06-42d6-917d-139d1358f1ae None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:12.476909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f0fbf-e9e5-471c-9ca5-d23bb9335673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:12.476909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f0fbf-e9e5-471c-9ca5-d23bb9335673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:12.476909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f0fbf-e9e5-471c-9ca5-d23bb9335673 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:12.488552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 270/560] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:12.496528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee7d59-47f4-481a-9744-4e2dd2da5bbb None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.496528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee7d59-47f4-481a-9744-4e2dd2da5bbb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.496528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee7d59-47f4-481a-9744-4e2dd2da5bbb None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.496528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 271/561] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:12.497156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee7d59-47f4-481a-9744-4e2dd2da5bbb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:12.572601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe3ae-7e06-42d6-917d-139d1358f1ae None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:12.604047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee7d59-47f4-481a-9744-4e2dd2da5bbb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:12.604237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c3720-4a82-445e-b79d-091a321a2b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:12.604237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c3720-4a82-445e-b79d-091a321a2b6c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.604237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c3720-4a82-445e-b79d-091a321a2b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.604237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c3720-4a82-445e-b79d-091a321a2b6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.620948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c3720-4a82-445e-b79d-091a321a2b6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ae8c3720-4a82-445e-b79d-091a321a2b6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:12.636527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:12.637684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 291/562] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:12.639248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c3720-4a82-445e-b79d-091a321a2b6c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:12.640334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b7922-ed4e-475d-b8e5-63b3a7315b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:12.640334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b7922-ed4e-475d-b8e5-63b3a7315b7f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.640334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b7922-ed4e-475d-b8e5-63b3a7315b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.640334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b7922-ed4e-475d-b8e5-63b3a7315b7f None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.641141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b7922-ed4e-475d-b8e5-63b3a7315b7f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:12.641141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b7922-ed4e-475d-b8e5-63b3a7315b7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:12.642613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b7922-ed4e-475d-b8e5-63b3a7315b7f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:12.663191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 272/563] 10.222.0.50 () {56 vars in 1135 bytes} [Fri Apr 29 18:30:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:12.664598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecfc05-8ade-4421-9f07-0517b59fdc90 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.665078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecfc05-8ade-4421-9f07-0517b59fdc90 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.665507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecfc05-8ade-4421-9f07-0517b59fdc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.689196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d6ecfc05-8ade-4421-9f07-0517b59fdc90 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:12.690392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6ecfc05-8ade-4421-9f07-0517b59fdc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:12.761240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c3720-4a82-445e-b79d-091a321a2b6c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:12.790170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 273/564] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:12.791010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecfc05-8ade-4421-9f07-0517b59fdc90 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:12.820208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af151ae-588f-44d5-9552-01927e584260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:12.821420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af151ae-588f-44d5-9552-01927e584260 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.821918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af151ae-588f-44d5-9552-01927e584260 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.822097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af151ae-588f-44d5-9552-01927e584260 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.830105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af151ae-588f-44d5-9552-01927e584260 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:12.831127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af151ae-588f-44d5-9552-01927e584260 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2af151ae-588f-44d5-9552-01927e584260 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:12.848275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:12.849294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:12.850262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:12.850262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:12.850262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:12.850262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:12.850262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:12.850262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:12.850262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:12.850262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:12.850262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 292/565] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:12.850813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af151ae-588f-44d5-9552-01927e584260 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:12.895266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecfc05-8ade-4421-9f07-0517b59fdc90 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:12.929217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b8ade-d75d-449d-9064-694ba735666a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:12.929887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b8ade-d75d-449d-9064-694ba735666a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:12.930160 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b8ade-d75d-449d-9064-694ba735666a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:12.930426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b8ade-d75d-449d-9064-694ba735666a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:12.931227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b8ade-d75d-449d-9064-694ba735666a None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:12.931526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b8ade-d75d-449d-9064-694ba735666a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:12.932926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b8ade-d75d-449d-9064-694ba735666a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:12.940705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 274/566] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:12.941483 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b8ade-d75d-449d-9064-694ba735666a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:12.960185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af151ae-588f-44d5-9552-01927e584260 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.001734 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04febd9-c6c3-45fd-a127-b66023524050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.003128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04febd9-c6c3-45fd-a127-b66023524050 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.004333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04febd9-c6c3-45fd-a127-b66023524050 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.004652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04febd9-c6c3-45fd-a127-b66023524050 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.040390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b8ade-d75d-449d-9064-694ba735666a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.070617 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04febd9-c6c3-45fd-a127-b66023524050 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:13.071497 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a2b00-093a-4b5b-95f3-eabf0250586a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.072088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04febd9-c6c3-45fd-a127-b66023524050 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:13.072364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a2b00-093a-4b5b-95f3-eabf0250586a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.072671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a2b00-093a-4b5b-95f3-eabf0250586a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.072936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a2b00-093a-4b5b-95f3-eabf0250586a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.083012 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a2b00-093a-4b5b-95f3-eabf0250586a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6a1a2b00-093a-4b5b-95f3-eabf0250586a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:13.092238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:13.093766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 275/567] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:13.094689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a2b00-093a-4b5b-95f3-eabf0250586a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:13.170662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 293/568] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:13.171482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04febd9-c6c3-45fd-a127-b66023524050 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:13.193488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a2b00-093a-4b5b-95f3-eabf0250586a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.226494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d7c62-5359-4674-9aa7-135bddb40ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.227143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d7c62-5359-4674-9aa7-135bddb40ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.227552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d7c62-5359-4674-9aa7-135bddb40ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.228043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d7c62-5359-4674-9aa7-135bddb40ed5 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.228748 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d7c62-5359-4674-9aa7-135bddb40ed5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:13.229054 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d7c62-5359-4674-9aa7-135bddb40ed5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:13.230473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d7c62-5359-4674-9aa7-135bddb40ed5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:13.241796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 276/569] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:13.243217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c192ad-1c23-411a-9eb0-5bf0223ed87c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.243391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c192ad-1c23-411a-9eb0-5bf0223ed87c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.243570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c192ad-1c23-411a-9eb0-5bf0223ed87c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.245171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 277/570] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:13.245799 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c192ad-1c23-411a-9eb0-5bf0223ed87c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:13.273519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04febd9-c6c3-45fd-a127-b66023524050 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.307348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdcee9-8bfc-421d-aaf0-cbe0716e7c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.308143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdcee9-8bfc-421d-aaf0-cbe0716e7c39 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.308413 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdcee9-8bfc-421d-aaf0-cbe0716e7c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.308721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdcee9-8bfc-421d-aaf0-cbe0716e7c39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.318870 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdcee9-8bfc-421d-aaf0-cbe0716e7c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:13.320321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdcee9-8bfc-421d-aaf0-cbe0716e7c39 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-7ebdcee9-8bfc-421d-aaf0-cbe0716e7c39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:13.341002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:13.341996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:13.344053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:13.344053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:13.344053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:13.344053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:13.344053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 294/571] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:13.344053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdcee9-8bfc-421d-aaf0-cbe0716e7c39 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:13.357424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c192ad-1c23-411a-9eb0-5bf0223ed87c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.397771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8839239-9916-4766-9de9-b72fdeb87333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.398768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8839239-9916-4766-9de9-b72fdeb87333 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.399212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8839239-9916-4766-9de9-b72fdeb87333 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.399812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8839239-9916-4766-9de9-b72fdeb87333 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.401340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8839239-9916-4766-9de9-b72fdeb87333 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:13.401825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8839239-9916-4766-9de9-b72fdeb87333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:13.403645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8839239-9916-4766-9de9-b72fdeb87333 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:13.413418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 278/572] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:13.414188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8839239-9916-4766-9de9-b72fdeb87333 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:13.439857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdcee9-8bfc-421d-aaf0-cbe0716e7c39 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.475148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421b298-c3f4-45d8-b416-13629827fda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.475940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b298-c3f4-45d8-b416-13629827fda2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.476219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b298-c3f4-45d8-b416-13629827fda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.476516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b298-c3f4-45d8-b416-13629827fda2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.486651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421b298-c3f4-45d8-b416-13629827fda2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:13.488120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421b298-c3f4-45d8-b416-13629827fda2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6421b298-c3f4-45d8-b416-13629827fda2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:13.505424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:13.506333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:13.507294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:13.507294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:13.507294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:13.507294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:13.507294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 295/573] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:13.507565 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8839239-9916-4766-9de9-b72fdeb87333 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.508072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50cdca-7583-46fb-b9dd-2334ea3dc420 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.508350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50cdca-7583-46fb-b9dd-2334ea3dc420 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.508682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50cdca-7583-46fb-b9dd-2334ea3dc420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.526661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-da50cdca-7583-46fb-b9dd-2334ea3dc420 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:13.527837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-da50cdca-7583-46fb-b9dd-2334ea3dc420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:13.543322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c5911-0df5-448e-922d-ff2ba9bb9959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.544193 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c5911-0df5-448e-922d-ff2ba9bb9959 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.544519 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c5911-0df5-448e-922d-ff2ba9bb9959 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.544781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c5911-0df5-448e-922d-ff2ba9bb9959 None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/341358254d884b278d7fa19d95c2dc2d/roles/4243cffaa7424ff99b0e23fc4940bae5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.559036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c5911-0df5-448e-922d-ff2ba9bb9959 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=341358254d884b278d7fa19d95c2dc2d, role_id=4243cffaa7424ff99b0e23fc4940bae5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:13.560509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c5911-0df5-448e-922d-ff2ba9bb9959 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:13.573628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 279/574] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:30:13 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/341358254d884b278d7fa19d95c2dc2d/roles/4243cffaa7424ff99b0e23fc4940bae5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:30:13.574833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c5911-0df5-448e-922d-ff2ba9bb9959 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:13.608609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 296/575] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:13.608609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50cdca-7583-46fb-b9dd-2334ea3dc420 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:13.700323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c5911-0df5-448e-922d-ff2ba9bb9959 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.724260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50cdca-7583-46fb-b9dd-2334ea3dc420 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.742120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3631c8f-d3a2-4bd5-ab8a-933263e651db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.743447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3631c8f-d3a2-4bd5-ab8a-933263e651db None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.743819 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3631c8f-d3a2-4bd5-ab8a-933263e651db None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.744262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3631c8f-d3a2-4bd5-ab8a-933263e651db None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.745940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3631c8f-d3a2-4bd5-ab8a-933263e651db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:13.746426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3631c8f-d3a2-4bd5-ab8a-933263e651db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:13.748280 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3631c8f-d3a2-4bd5-ab8a-933263e651db None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:13.762920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c0378-3b80-43ba-b104-6a4401a45371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.764235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0378-3b80-43ba-b104-6a4401a45371 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.764619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0378-3b80-43ba-b104-6a4401a45371 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.764923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0378-3b80-43ba-b104-6a4401a45371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.766273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 280/576] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:13.767556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3631c8f-d3a2-4bd5-ab8a-933263e651db None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:13.778492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c0378-3b80-43ba-b104-6a4401a45371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:13.780112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c0378-3b80-43ba-b104-6a4401a45371 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-e26c0378-3b80-43ba-b104-6a4401a45371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:13.798915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:13.799917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:13.800887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:13.800887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:13.800887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:13.800887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:13.800887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 297/577] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:13.801725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c0378-3b80-43ba-b104-6a4401a45371 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:13.890913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3631c8f-d3a2-4bd5-ab8a-933263e651db None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.899893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c0378-3b80-43ba-b104-6a4401a45371 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:13.930055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cef53-32a9-49b9-a56b-f7ba8234db78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.931262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cef53-32a9-49b9-a56b-f7ba8234db78 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.931772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cef53-32a9-49b9-a56b-f7ba8234db78 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.932150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cef53-32a9-49b9-a56b-f7ba8234db78 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.933810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cef53-32a9-49b9-a56b-f7ba8234db78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:13.934281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cef53-32a9-49b9-a56b-f7ba8234db78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:13.934849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ec1c5-01d4-40fa-bdd9-fba05aa6d026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:13.935654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ec1c5-01d4-40fa-bdd9-fba05aa6d026 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:13.935935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ec1c5-01d4-40fa-bdd9-fba05aa6d026 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:13.936096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cef53-32a9-49b9-a56b-f7ba8234db78 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:13.936198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ec1c5-01d4-40fa-bdd9-fba05aa6d026 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:13.946543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ec1c5-01d4-40fa-bdd9-fba05aa6d026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:13.954660 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 281/578] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:13.954660 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cef53-32a9-49b9-a56b-f7ba8234db78 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6d1ec1c5-01d4-40fa-bdd9-fba05aa6d026 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:13.954914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:13.955989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:13.956871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 298/579] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:13.957580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ec1c5-01d4-40fa-bdd9-fba05aa6d026 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.053773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ec1c5-01d4-40fa-bdd9-fba05aa6d026 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.073392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cef53-32a9-49b9-a56b-f7ba8234db78 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.086036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e991005-d880-4fba-9381-93d5f207075a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.086682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e991005-d880-4fba-9381-93d5f207075a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.087077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e991005-d880-4fba-9381-93d5f207075a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.087340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e991005-d880-4fba-9381-93d5f207075a None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.088172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e991005-d880-4fba-9381-93d5f207075a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:14.088483 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e991005-d880-4fba-9381-93d5f207075a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.089880 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e991005-d880-4fba-9381-93d5f207075a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.106686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 299/580] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:14.106686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e991005-d880-4fba-9381-93d5f207075a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.114029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c717d-88ec-4ebb-bed4-fb07363730c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.115121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c717d-88ec-4ebb-bed4-fb07363730c2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.115560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c717d-88ec-4ebb-bed4-fb07363730c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.115977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c717d-88ec-4ebb-bed4-fb07363730c2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.125549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c717d-88ec-4ebb-bed4-fb07363730c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.127465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c717d-88ec-4ebb-bed4-fb07363730c2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-403c717d-88ec-4ebb-bed4-fb07363730c2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:14.143310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:14.144454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:14.145450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 282/581] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:14.146061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c717d-88ec-4ebb-bed4-fb07363730c2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.208557 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e991005-d880-4fba-9381-93d5f207075a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.247972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732696d-9158-4dbc-8f07-aafd15d10ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.248540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c717d-88ec-4ebb-bed4-fb07363730c2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.248812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732696d-9158-4dbc-8f07-aafd15d10ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.249089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732696d-9158-4dbc-8f07-aafd15d10ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.249357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732696d-9158-4dbc-8f07-aafd15d10ac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.256024 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732696d-9158-4dbc-8f07-aafd15d10ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.257510 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732696d-9158-4dbc-8f07-aafd15d10ac6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d732696d-9158-4dbc-8f07-aafd15d10ac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:14.270985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:14.271913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:14.272818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:14.272818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:14.272818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:14.272818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:14.272818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:14.272818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:14.272818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:14.272818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:14.272818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 300/582] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:14.273328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732696d-9158-4dbc-8f07-aafd15d10ac6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.290048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bacfe-3c8b-4dd4-8020-f7dda81279cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.290743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bacfe-3c8b-4dd4-8020-f7dda81279cf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.291008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bacfe-3c8b-4dd4-8020-f7dda81279cf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.291282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bacfe-3c8b-4dd4-8020-f7dda81279cf None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.292030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bacfe-3c8b-4dd4-8020-f7dda81279cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:14.292330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bacfe-3c8b-4dd4-8020-f7dda81279cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.293739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bacfe-3c8b-4dd4-8020-f7dda81279cf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.306087 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 283/583] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:14.306087 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bacfe-3c8b-4dd4-8020-f7dda81279cf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.377074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732696d-9158-4dbc-8f07-aafd15d10ac6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.404289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bacfe-3c8b-4dd4-8020-f7dda81279cf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.410380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce96d1-5a71-47c3-9b21-19a7ef138314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.411192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce96d1-5a71-47c3-9b21-19a7ef138314 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.411478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce96d1-5a71-47c3-9b21-19a7ef138314 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.411738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce96d1-5a71-47c3-9b21-19a7ef138314 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.419461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce96d1-5a71-47c3-9b21-19a7ef138314 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.420936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce96d1-5a71-47c3-9b21-19a7ef138314 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-a3ce96d1-5a71-47c3-9b21-19a7ef138314 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:14.434819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:14.435858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 301/584] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:14.436837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce96d1-5a71-47c3-9b21-19a7ef138314 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.445635 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81047b8-1d36-4537-befa-099a6bc64984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.446419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81047b8-1d36-4537-befa-099a6bc64984 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.446761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81047b8-1d36-4537-befa-099a6bc64984 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.447038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81047b8-1d36-4537-befa-099a6bc64984 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.447883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81047b8-1d36-4537-befa-099a6bc64984 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:14.448187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81047b8-1d36-4537-befa-099a6bc64984 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.449634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81047b8-1d36-4537-befa-099a6bc64984 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.471575 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 284/585] 10.222.0.50 () {56 vars in 1137 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:14.471575 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81047b8-1d36-4537-befa-099a6bc64984 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.539180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce96d1-5a71-47c3-9b21-19a7ef138314 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.560992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81047b8-1d36-4537-befa-099a6bc64984 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.570727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff11868-55d0-4f4e-a41e-854acd0bfbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.571424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff11868-55d0-4f4e-a41e-854acd0bfbec None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.571716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff11868-55d0-4f4e-a41e-854acd0bfbec None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.572018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff11868-55d0-4f4e-a41e-854acd0bfbec None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.572797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff11868-55d0-4f4e-a41e-854acd0bfbec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:14.573102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff11868-55d0-4f4e-a41e-854acd0bfbec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.574534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff11868-55d0-4f4e-a41e-854acd0bfbec None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.591251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 302/586] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:14.591251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff11868-55d0-4f4e-a41e-854acd0bfbec None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.594777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5128dc-1259-4c91-9289-44faa68db8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.595590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5128dc-1259-4c91-9289-44faa68db8cc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.595878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5128dc-1259-4c91-9289-44faa68db8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.596141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5128dc-1259-4c91-9289-44faa68db8cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.611311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5128dc-1259-4c91-9289-44faa68db8cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.612876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5128dc-1259-4c91-9289-44faa68db8cc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4c5128dc-1259-4c91-9289-44faa68db8cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:14.634330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:14.635258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:14.636166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:14.636166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:14.636166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:14.636166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:14.640179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 285/587] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:14.641062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5128dc-1259-4c91-9289-44faa68db8cc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.683027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff11868-55d0-4f4e-a41e-854acd0bfbec None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.716793 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3c29b-6b49-4371-b980-3507db26c509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.717835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3c29b-6b49-4371-b980-3507db26c509 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.718114 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3c29b-6b49-4371-b980-3507db26c509 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.718383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3c29b-6b49-4371-b980-3507db26c509 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.719216 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3c29b-6b49-4371-b980-3507db26c509 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:14.719518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3c29b-6b49-4371-b980-3507db26c509 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.720975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3c29b-6b49-4371-b980-3507db26c509 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.738087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 303/588] 10.222.0.50 () {56 vars in 1137 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:14.738901 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3c29b-6b49-4371-b980-3507db26c509 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.740358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5128dc-1259-4c91-9289-44faa68db8cc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.777025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e4684-65a5-40f8-9077-47a76b21c704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.777831 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e4684-65a5-40f8-9077-47a76b21c704 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.778111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e4684-65a5-40f8-9077-47a76b21c704 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.778381 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e4684-65a5-40f8-9077-47a76b21c704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.788328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e4684-65a5-40f8-9077-47a76b21c704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.789912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e4684-65a5-40f8-9077-47a76b21c704 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-777e4684-65a5-40f8-9077-47a76b21c704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:14.808037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:14.809201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:14.814903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:14.814903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:14.814903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:14.814903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:14.814903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 286/589] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:14.814903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e4684-65a5-40f8-9077-47a76b21c704 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.833584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3c29b-6b49-4371-b980-3507db26c509 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.866398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1d4ec-a69c-4690-8a8a-0760134a5b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.867056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1d4ec-a69c-4690-8a8a-0760134a5b15 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.867551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1d4ec-a69c-4690-8a8a-0760134a5b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.867670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1d4ec-a69c-4690-8a8a-0760134a5b15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.868858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1d4ec-a69c-4690-8a8a-0760134a5b15 None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=4cf46333cd034f5dbd0b78503de1f8e1) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:14.869065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1d4ec-a69c-4690-8a8a-0760134a5b15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.870723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1d4ec-a69c-4690-8a8a-0760134a5b15 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.875887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 304/590] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/role_assignments?user.id=4cf46333cd034f5dbd0b78503de1f8e1&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:14.876980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1d4ec-a69c-4690-8a8a-0760134a5b15 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.914001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e4684-65a5-40f8-9077-47a76b21c704 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:14.951935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ec04b-9040-46c0-964e-459ee58f469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:14.952672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ec04b-9040-46c0-964e-459ee58f469b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:14.952954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ec04b-9040-46c0-964e-459ee58f469b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:14.953224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ec04b-9040-46c0-964e-459ee58f469b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:14.954171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ec04b-9040-46c0-964e-459ee58f469b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:14.954283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ec04b-9040-46c0-964e-459ee58f469b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:14.955666 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ec04b-9040-46c0-964e-459ee58f469b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:14.969782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 287/591] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:14.970512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ec04b-9040-46c0-964e-459ee58f469b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:14.992898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1d4ec-a69c-4690-8a8a-0760134a5b15 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.030107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0ec68-8d88-4cac-b9d5-2c4461f2b9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.031309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0ec68-8d88-4cac-b9d5-2c4461f2b9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.031836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0ec68-8d88-4cac-b9d5-2c4461f2b9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.032324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0ec68-8d88-4cac-b9d5-2c4461f2b9b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.045009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0ec68-8d88-4cac-b9d5-2c4461f2b9b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.046872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0ec68-8d88-4cac-b9d5-2c4461f2b9b3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-99c0ec68-8d88-4cac-b9d5-2c4461f2b9b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:15.066139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:15.067530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:15.068465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:15.068465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:15.068465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:15.068465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:15.072549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 305/592] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:15.076399 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ec04b-9040-46c0-964e-459ee58f469b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.077339 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0ec68-8d88-4cac-b9d5-2c4461f2b9b3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.121624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d34a0-f7f7-4ab4-ab01-3bf9a5fc00ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.122391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d34a0-f7f7-4ab4-ab01-3bf9a5fc00ef None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.122743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d34a0-f7f7-4ab4-ab01-3bf9a5fc00ef None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.123086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d34a0-f7f7-4ab4-ab01-3bf9a5fc00ef None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.123881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d34a0-f7f7-4ab4-ab01-3bf9a5fc00ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:15.124276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d34a0-f7f7-4ab4-ab01-3bf9a5fc00ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.125829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d34a0-f7f7-4ab4-ab01-3bf9a5fc00ef None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.138584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 288/593] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:15.139598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d34a0-f7f7-4ab4-ab01-3bf9a5fc00ef None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.191640 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0ec68-8d88-4cac-b9d5-2c4461f2b9b3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.225314 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbce6c-dfa7-4a5a-a124-b39f47473c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.226053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbce6c-dfa7-4a5a-a124-b39f47473c09 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.226333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbce6c-dfa7-4a5a-a124-b39f47473c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.226596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbce6c-dfa7-4a5a-a124-b39f47473c09 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.234804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbce6c-dfa7-4a5a-a124-b39f47473c09 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.236257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbce6c-dfa7-4a5a-a124-b39f47473c09 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.242390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d34a0-f7f7-4ab4-ab01-3bf9a5fc00ef None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-81bbce6c-dfa7-4a5a-a124-b39f47473c09 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:15.250950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:15.251915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:15.252905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 306/594] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:15.253514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbce6c-dfa7-4a5a-a124-b39f47473c09 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.278845 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06208-c8f8-4254-9b6f-e3ba62f7fde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.279549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06208-c8f8-4254-9b6f-e3ba62f7fde7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.279827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06208-c8f8-4254-9b6f-e3ba62f7fde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.280096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06208-c8f8-4254-9b6f-e3ba62f7fde7 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.280859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd06208-c8f8-4254-9b6f-e3ba62f7fde7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:15.281162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd06208-c8f8-4254-9b6f-e3ba62f7fde7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.282630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd06208-c8f8-4254-9b6f-e3ba62f7fde7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.294578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 289/595] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:15.295409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06208-c8f8-4254-9b6f-e3ba62f7fde7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.357830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbce6c-dfa7-4a5a-a124-b39f47473c09 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.391041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628ba6c-8114-4938-8084-37d2c2f4a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.391903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628ba6c-8114-4938-8084-37d2c2f4a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.392184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628ba6c-8114-4938-8084-37d2c2f4a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.392459 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628ba6c-8114-4938-8084-37d2c2f4a7c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.395909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06208-c8f8-4254-9b6f-e3ba62f7fde7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.399696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628ba6c-8114-4938-8084-37d2c2f4a7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.401228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628ba6c-8114-4938-8084-37d2c2f4a7c3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-3628ba6c-8114-4938-8084-37d2c2f4a7c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:15.416096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:15.424099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:15.426229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:15.426229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:15.426229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:15.426229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:15.426229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:15.426229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:15.426229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:15.426229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:15.426229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 307/596] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:15.426229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628ba6c-8114-4938-8084-37d2c2f4a7c3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.436536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e7c22-9f4b-41fc-8d2e-ba1843c6ed04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.436536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e7c22-9f4b-41fc-8d2e-ba1843c6ed04 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.436536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e7c22-9f4b-41fc-8d2e-ba1843c6ed04 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.436536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e7c22-9f4b-41fc-8d2e-ba1843c6ed04 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.437236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e7c22-9f4b-41fc-8d2e-ba1843c6ed04 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:15.437236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e7c22-9f4b-41fc-8d2e-ba1843c6ed04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.437236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e7c22-9f4b-41fc-8d2e-ba1843c6ed04 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.458552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 290/597] 10.222.0.50 () {56 vars in 1139 bytes} [Fri Apr 29 18:30:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:15.458552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e7c22-9f4b-41fc-8d2e-ba1843c6ed04 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.524269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628ba6c-8114-4938-8084-37d2c2f4a7c3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.556531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e7c22-9f4b-41fc-8d2e-ba1843c6ed04 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.557162 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad777e7-dac2-4540-90e0-1603bf960743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.557972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad777e7-dac2-4540-90e0-1603bf960743 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.558255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad777e7-dac2-4540-90e0-1603bf960743 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.559276 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad777e7-dac2-4540-90e0-1603bf960743 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.565637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad777e7-dac2-4540-90e0-1603bf960743 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.567086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad777e7-dac2-4540-90e0-1603bf960743 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-4ad777e7-dac2-4540-90e0-1603bf960743 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:15.581385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:15.582514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:15.583537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:15.583537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:15.583537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:15.583537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:15.583537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:15.583537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:15.583537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:15.583537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:15.587925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 308/598] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:15.588726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad777e7-dac2-4540-90e0-1603bf960743 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.596155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad37708-64a0-4103-8f47-d94dbcdbd709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.597071 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad37708-64a0-4103-8f47-d94dbcdbd709 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.597487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad37708-64a0-4103-8f47-d94dbcdbd709 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.597905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad37708-64a0-4103-8f47-d94dbcdbd709 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.598832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad37708-64a0-4103-8f47-d94dbcdbd709 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:15.599273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad37708-64a0-4103-8f47-d94dbcdbd709 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.600879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad37708-64a0-4103-8f47-d94dbcdbd709 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.613373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 291/599] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:15.614408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad37708-64a0-4103-8f47-d94dbcdbd709 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.698930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad777e7-dac2-4540-90e0-1603bf960743 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.720117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad37708-64a0-4103-8f47-d94dbcdbd709 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.733379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4031abd-6af4-48dd-b367-e67c6b9d9fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.734278 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4031abd-6af4-48dd-b367-e67c6b9d9fae None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.734634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4031abd-6af4-48dd-b367-e67c6b9d9fae None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.735012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4031abd-6af4-48dd-b367-e67c6b9d9fae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.747765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4031abd-6af4-48dd-b367-e67c6b9d9fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.749399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4031abd-6af4-48dd-b367-e67c6b9d9fae None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.759552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780b336-aeae-4094-9d53-2f9b5a3bbd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.760879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780b336-aeae-4094-9d53-2f9b5a3bbd41 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.761364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780b336-aeae-4094-9d53-2f9b5a3bbd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.761836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780b336-aeae-4094-9d53-2f9b5a3bbd41 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.762795 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780b336-aeae-4094-9d53-2f9b5a3bbd41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:15.763250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780b336-aeae-4094-9d53-2f9b5a3bbd41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.764944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780b336-aeae-4094-9d53-2f9b5a3bbd41 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-c4031abd-6af4-48dd-b367-e67c6b9d9fae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:15.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:15.768368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:15.769503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:15.769503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:15.769503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:15.769503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:15.771140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 309/600] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:15.772086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4031abd-6af4-48dd-b367-e67c6b9d9fae None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.783224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 292/601] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:15.784406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780b336-aeae-4094-9d53-2f9b5a3bbd41 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.887501 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4031abd-6af4-48dd-b367-e67c6b9d9fae None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.887989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780b336-aeae-4094-9d53-2f9b5a3bbd41 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:15.921117 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3791a7-6b90-4a9a-9ae0-e1f1d9ad9221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.922180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3791a7-6b90-4a9a-9ae0-e1f1d9ad9221 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.922512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44885d84-9611-4dda-bfb4-ad907d10d604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:15.922625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3791a7-6b90-4a9a-9ae0-e1f1d9ad9221 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.922978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3791a7-6b90-4a9a-9ae0-e1f1d9ad9221 None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/6552c41f1c654ef5bfdd940e3b807dae/roles/4243cffaa7424ff99b0e23fc4940bae5` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.923168 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44885d84-9611-4dda-bfb4-ad907d10d604 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.923444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44885d84-9611-4dda-bfb4-ad907d10d604 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.923712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44885d84-9611-4dda-bfb4-ad907d10d604 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.924425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44885d84-9611-4dda-bfb4-ad907d10d604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:15.924752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44885d84-9611-4dda-bfb4-ad907d10d604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.926101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44885d84-9611-4dda-bfb4-ad907d10d604 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.944075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 293/602] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:15.944530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3791a7-6b90-4a9a-9ae0-e1f1d9ad9221 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=6552c41f1c654ef5bfdd940e3b807dae, role_id=4243cffaa7424ff99b0e23fc4940bae5)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:15.945263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44885d84-9611-4dda-bfb4-ad907d10d604 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:15.947009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3791a7-6b90-4a9a-9ae0-e1f1d9ad9221 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:15.972605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 310/603] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:30:15 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/6552c41f1c654ef5bfdd940e3b807dae/roles/4243cffaa7424ff99b0e23fc4940bae5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:30:15.974612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e041c1-f78f-45eb-a8bb-e80167cd5a9c None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:15.974919 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e041c1-f78f-45eb-a8bb-e80167cd5a9c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:15.975192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e041c1-f78f-45eb-a8bb-e80167cd5a9c None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:15.981360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 311/604] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:15.981360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e041c1-f78f-45eb-a8bb-e80167cd5a9c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:16.082750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44885d84-9611-4dda-bfb4-ad907d10d604 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:16.100232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e041c1-f78f-45eb-a8bb-e80167cd5a9c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:16.122813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bc1d9-9e70-4f43-8f43-8379a68b7e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:16.124319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bc1d9-9e70-4f43-8f43-8379a68b7e42 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:16.124791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bc1d9-9e70-4f43-8f43-8379a68b7e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:16.125207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bc1d9-9e70-4f43-8f43-8379a68b7e42 None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/ae4fe640bb564a8abe872cb1ec74dc7d/roles/4243cffaa7424ff99b0e23fc4940bae5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:16.142113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bc1d9-9e70-4f43-8f43-8379a68b7e42 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=ae4fe640bb564a8abe872cb1ec74dc7d, role_id=4243cffaa7424ff99b0e23fc4940bae5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:16.143818 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bc1d9-9e70-4f43-8f43-8379a68b7e42 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:16.145015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4421ed1-562d-48ff-b1d9-cc2658972b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:16.145836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4421ed1-562d-48ff-b1d9-cc2658972b56 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:16.146112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4421ed1-562d-48ff-b1d9-cc2658972b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:16.146378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4421ed1-562d-48ff-b1d9-cc2658972b56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:16.154672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4421ed1-562d-48ff-b1d9-cc2658972b56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:16.156859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4421ed1-562d-48ff-b1d9-cc2658972b56 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:16.169670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 294/605] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:30:15 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/ae4fe640bb564a8abe872cb1ec74dc7d/roles/4243cffaa7424ff99b0e23fc4940bae5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:30:16.171517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56062b0f-cc4a-459b-b737-410ce070b07f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:16.171990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56062b0f-cc4a-459b-b737-410ce070b07f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f4421ed1-562d-48ff-b1d9-cc2658972b56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:16.172291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:16.173365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:16.174355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:16.174355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:16.174355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:16.174355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:16.174355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:16.174355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:16.174355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:16.174355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:16.174985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56062b0f-cc4a-459b-b737-410ce070b07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:16.180575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 312/606] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:16.180891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4421ed1-562d-48ff-b1d9-cc2658972b56 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:16.221457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-56062b0f-cc4a-459b-b737-410ce070b07f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:16.224585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-56062b0f-cc4a-459b-b737-410ce070b07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:16.339756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4421ed1-562d-48ff-b1d9-cc2658972b56 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:16.346654 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 295/607] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:16.386522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56062b0f-cc4a-459b-b737-410ce070b07f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:16.398569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56ff3d-42b3-45dd-93d7-9c8dcdd037aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:16.399550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56ff3d-42b3-45dd-93d7-9c8dcdd037aa None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:16.401358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56ff3d-42b3-45dd-93d7-9c8dcdd037aa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:16.401745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56ff3d-42b3-45dd-93d7-9c8dcdd037aa None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:16.402681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56ff3d-42b3-45dd-93d7-9c8dcdd037aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:16.404748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56ff3d-42b3-45dd-93d7-9c8dcdd037aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:16.406334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56ff3d-42b3-45dd-93d7-9c8dcdd037aa None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:16.443013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 313/608] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:16.447611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56ff3d-42b3-45dd-93d7-9c8dcdd037aa None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:16.586220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56062b0f-cc4a-459b-b737-410ce070b07f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:16.605965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56ff3d-42b3-45dd-93d7-9c8dcdd037aa None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:16.673299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afc607-c6cf-4da2-958c-a807e8554ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:16.674111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afc607-c6cf-4da2-958c-a807e8554ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:16.680812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afc607-c6cf-4da2-958c-a807e8554ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:16.680812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afc607-c6cf-4da2-958c-a807e8554ac9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:16.681040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77c5c4-b731-49e3-9da6-4da9c8432c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:16.684514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77c5c4-b731-49e3-9da6-4da9c8432c18 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:16.684514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77c5c4-b731-49e3-9da6-4da9c8432c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:16.684514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77c5c4-b731-49e3-9da6-4da9c8432c18 None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/07849b2925334763b119111ed52dd34c/roles/4243cffaa7424ff99b0e23fc4940bae5` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:16.693097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afc607-c6cf-4da2-958c-a807e8554ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e9afc607-c6cf-4da2-958c-a807e8554ac9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:16.705934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:16.706935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:16.707773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:16.714566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77c5c4-b731-49e3-9da6-4da9c8432c18 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=07849b2925334763b119111ed52dd34c, role_id=4243cffaa7424ff99b0e23fc4940bae5)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:16.716022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77c5c4-b731-49e3-9da6-4da9c8432c18 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:16.718362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 296/609] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:16.719240 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afc607-c6cf-4da2-958c-a807e8554ac9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:16.759920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 314/610] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:30:16 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/07849b2925334763b119111ed52dd34c/roles/4243cffaa7424ff99b0e23fc4940bae5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:30:16.862067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afc607-c6cf-4da2-958c-a807e8554ac9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:16.907017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bef21-cea5-4b90-a4f6-b22c78b56ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:16.907736 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bef21-cea5-4b90-a4f6-b22c78b56ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:16.908072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bef21-cea5-4b90-a4f6-b22c78b56ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:16.908348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bef21-cea5-4b90-a4f6-b22c78b56ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:16.909185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bef21-cea5-4b90-a4f6-b22c78b56ce1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:16.909491 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bef21-cea5-4b90-a4f6-b22c78b56ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:16.910899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bef21-cea5-4b90-a4f6-b22c78b56ce1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:16.930338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 297/611] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:16.935633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77c5c4-b731-49e3-9da6-4da9c8432c18 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:17.084733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77c5c4-b731-49e3-9da6-4da9c8432c18 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:17.122618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b8623-757c-44b2-ac69-759cf6c28ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:17.123386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b8623-757c-44b2-ac69-759cf6c28ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:17.123613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b8623-757c-44b2-ac69-759cf6c28ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:17.123878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b8623-757c-44b2-ac69-759cf6c28ad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:17.145623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b8623-757c-44b2-ac69-759cf6c28ad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:17.147104 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b8623-757c-44b2-ac69-759cf6c28ad3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f48b8623-757c-44b2-ac69-759cf6c28ad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:17.176659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:17.177626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:17.178526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:17.178526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:17.178526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:17.178526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:17.182364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 315/612] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:17.185849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bef21-cea5-4b90-a4f6-b22c78b56ce1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:17.299110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bef21-cea5-4b90-a4f6-b22c78b56ce1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:17.339726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377494d-cc9f-4c87-bcef-61303206d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:17.340467 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377494d-cc9f-4c87-bcef-61303206d1dc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:17.340778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377494d-cc9f-4c87-bcef-61303206d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:17.341054 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377494d-cc9f-4c87-bcef-61303206d1dc None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:17.341800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377494d-cc9f-4c87-bcef-61303206d1dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:17.342109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377494d-cc9f-4c87-bcef-61303206d1dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:17.343848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377494d-cc9f-4c87-bcef-61303206d1dc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:17.368256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 298/613] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:17.374295 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b8623-757c-44b2-ac69-759cf6c28ad3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:17.491826 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b8623-757c-44b2-ac69-759cf6c28ad3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:17.534083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46883d5a-5911-4b31-bdf5-fbf5847b7052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:17.534898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46883d5a-5911-4b31-bdf5-fbf5847b7052 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:17.535181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46883d5a-5911-4b31-bdf5-fbf5847b7052 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:17.535454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46883d5a-5911-4b31-bdf5-fbf5847b7052 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:17.543060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46883d5a-5911-4b31-bdf5-fbf5847b7052 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:17.544525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46883d5a-5911-4b31-bdf5-fbf5847b7052 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:17.548120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dda5a-c729-4b75-b62b-c15c77ecd208 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:17.548407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dda5a-c729-4b75-b62b-c15c77ecd208 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:17.548732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dda5a-c729-4b75-b62b-c15c77ecd208 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:17.550915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 299/614] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:17.557774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d8ae0-0be8-4634-afc1-5813dd50b578 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:17.558362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d8ae0-0be8-4634-afc1-5813dd50b578 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:17.558831 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d8ae0-0be8-4634-afc1-5813dd50b578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-46883d5a-5911-4b31-bdf5-fbf5847b7052 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:17.582746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:17.583760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:17.584754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:17.584754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:17.584754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:17.584754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:17.584754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:17.584754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:17.584754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:17.584754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:17.584754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:17.584754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:17.591800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ef0d8ae0-0be8-4634-afc1-5813dd50b578 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:17.593609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ef0d8ae0-0be8-4634-afc1-5813dd50b578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:17.593838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 316/615] 10.222.0.50 () {56 vars in 1093 bytes} [Fri Apr 29 18:30:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:17.594820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46883d5a-5911-4b31-bdf5-fbf5847b7052 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:17.745792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 300/616] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:17.763134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46883d5a-5911-4b31-bdf5-fbf5847b7052 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:17.779762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d8ae0-0be8-4634-afc1-5813dd50b578 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:17.827741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85369793-1b56-488b-9119-e8f8be673acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:17.828910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85369793-1b56-488b-9119-e8f8be673acb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:17.830230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85369793-1b56-488b-9119-e8f8be673acb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:17.830892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85369793-1b56-488b-9119-e8f8be673acb None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:17.833257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85369793-1b56-488b-9119-e8f8be673acb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:17.833871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85369793-1b56-488b-9119-e8f8be673acb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:17.836962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85369793-1b56-488b-9119-e8f8be673acb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:17.873097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 317/617] 10.222.0.50 () {56 vars in 1133 bytes} [Fri Apr 29 18:30:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:17.877550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85369793-1b56-488b-9119-e8f8be673acb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:17.996518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d8ae0-0be8-4634-afc1-5813dd50b578 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:18.004862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85369793-1b56-488b-9119-e8f8be673acb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:18.040797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8b220-efd2-41f4-8f25-21e0db18c782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:18.041732 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8b220-efd2-41f4-8f25-21e0db18c782 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:18.042017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8b220-efd2-41f4-8f25-21e0db18c782 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:18.042296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8b220-efd2-41f4-8f25-21e0db18c782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:18.109733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b11743-79ab-4566-8044-3562170cfb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:18.110552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b11743-79ab-4566-8044-3562170cfb29 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:18.110828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b11743-79ab-4566-8044-3562170cfb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:18.111096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b11743-79ab-4566-8044-3562170cfb29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:18.114279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8b220-efd2-41f4-8f25-21e0db18c782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:18.115742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8b220-efd2-41f4-8f25-21e0db18c782 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:18.137801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b11743-79ab-4566-8044-3562170cfb29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e8b11743-79ab-4566-8044-3562170cfb29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:18.149977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:18.150942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:18.151811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:18.155847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 301/618] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:18.156998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b11743-79ab-4566-8044-3562170cfb29 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:18.247952 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 318/619] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:18.251137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8b220-efd2-41f4-8f25-21e0db18c782 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:18.369095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8b220-efd2-41f4-8f25-21e0db18c782 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:18.371632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b11743-79ab-4566-8044-3562170cfb29 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:18.400828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a6b21-1b92-4146-8836-8bc1d1cfa575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:18.401553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a6b21-1b92-4146-8836-8bc1d1cfa575 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:18.407887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a6b21-1b92-4146-8836-8bc1d1cfa575 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:18.407887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a6b21-1b92-4146-8836-8bc1d1cfa575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:18.424529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a6b21-1b92-4146-8836-8bc1d1cfa575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:18.426340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a6b21-1b92-4146-8836-8bc1d1cfa575 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ce8a6b21-1b92-4146-8836-8bc1d1cfa575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:18.448143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:18.449122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:18.450020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:18.450020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:18.450020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:18.450020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:18.450020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 319/620] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:18.456169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a6b21-1b92-4146-8836-8bc1d1cfa575 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:18.462616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364497a-d76e-49b9-bb35-7170e72ecf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:18.463571 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364497a-d76e-49b9-bb35-7170e72ecf8b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:18.464053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364497a-d76e-49b9-bb35-7170e72ecf8b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:18.464631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364497a-d76e-49b9-bb35-7170e72ecf8b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:18.465600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364497a-d76e-49b9-bb35-7170e72ecf8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:18.466118 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364497a-d76e-49b9-bb35-7170e72ecf8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:18.468435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364497a-d76e-49b9-bb35-7170e72ecf8b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:18.485557 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 302/621] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:18.489278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364497a-d76e-49b9-bb35-7170e72ecf8b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:18.674430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a6b21-1b92-4146-8836-8bc1d1cfa575 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:18.710716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb0954-8126-4141-aac4-f88e4c4a7a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:18.711529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb0954-8126-4141-aac4-f88e4c4a7a73 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:18.711844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb0954-8126-4141-aac4-f88e4c4a7a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:18.712802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb0954-8126-4141-aac4-f88e4c4a7a73 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:18.713539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb0954-8126-4141-aac4-f88e4c4a7a73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:18.713845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb0954-8126-4141-aac4-f88e4c4a7a73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:18.714375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364497a-d76e-49b9-bb35-7170e72ecf8b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:18.715227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb0954-8126-4141-aac4-f88e4c4a7a73 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:18.729291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 320/622] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:18.735418 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb0954-8126-4141-aac4-f88e4c4a7a73 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:18.759106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c72d88-4a31-481f-8b56-1ab9670aace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:18.760159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c72d88-4a31-481f-8b56-1ab9670aace0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:18.760608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c72d88-4a31-481f-8b56-1ab9670aace0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:18.761022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c72d88-4a31-481f-8b56-1ab9670aace0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:18.780217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c72d88-4a31-481f-8b56-1ab9670aace0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:18.782067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c72d88-4a31-481f-8b56-1ab9670aace0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-94c72d88-4a31-481f-8b56-1ab9670aace0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:18.802116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:18.803244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:18.804242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:18.804242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:18.804242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:18.804242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:18.805222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 303/623] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:18.806747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780ce18-34cd-4dc5-86b7-9714d6bb4386 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:18.807615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780ce18-34cd-4dc5-86b7-9714d6bb4386 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:18.807615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780ce18-34cd-4dc5-86b7-9714d6bb4386 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:18.812627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 304/624] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:18.813674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780ce18-34cd-4dc5-86b7-9714d6bb4386 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:18.873999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb0954-8126-4141-aac4-f88e4c4a7a73 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:18.913950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d14a48-0edf-46e1-9b82-a36c17b718c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:18.915617 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d14a48-0edf-46e1-9b82-a36c17b718c3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:18.916074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d14a48-0edf-46e1-9b82-a36c17b718c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:18.916801 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d14a48-0edf-46e1-9b82-a36c17b718c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:18.926124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d14a48-0edf-46e1-9b82-a36c17b718c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:18.928080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d14a48-0edf-46e1-9b82-a36c17b718c3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:18.928234 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780ce18-34cd-4dc5-86b7-9714d6bb4386 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-89d14a48-0edf-46e1-9b82-a36c17b718c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:18.946542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:18.947710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:18.948624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:18.948624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:18.948624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:18.948624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:18.948624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:18.948624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:18.948624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:18.948624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:18.951378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 321/625] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:18.953375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7f5b-ba29-4652-91f6-a7008b2c84fa None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:18.953808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7f5b-ba29-4652-91f6-a7008b2c84fa None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:18.954465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7f5b-ba29-4652-91f6-a7008b2c84fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:18.965363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33a22-90b8-40bf-991b-bb28953845a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:18.966156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33a22-90b8-40bf-991b-bb28953845a4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:18.966502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33a22-90b8-40bf-991b-bb28953845a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:18.966891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33a22-90b8-40bf-991b-bb28953845a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:18.968744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c33a22-90b8-40bf-991b-bb28953845a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:18.969369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c33a22-90b8-40bf-991b-bb28953845a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:18.971458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c33a22-90b8-40bf-991b-bb28953845a4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:18.975553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ce1a7f5b-ba29-4652-91f6-a7008b2c84fa None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:18.976948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ce1a7f5b-ba29-4652-91f6-a7008b2c84fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:18.986291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 305/626] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:18.987662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33a22-90b8-40bf-991b-bb28953845a4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:19.059413 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 322/627] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:19.060279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a7f5b-ba29-4652-91f6-a7008b2c84fa None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:19.116622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33a22-90b8-40bf-991b-bb28953845a4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:19.150545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b13b97-fd83-4d37-b77c-e1b70d12f36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:19.151205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b13b97-fd83-4d37-b77c-e1b70d12f36b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.151485 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b13b97-fd83-4d37-b77c-e1b70d12f36b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.151752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b13b97-fd83-4d37-b77c-e1b70d12f36b None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.152556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b13b97-fd83-4d37-b77c-e1b70d12f36b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:19.152861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b13b97-fd83-4d37-b77c-e1b70d12f36b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:19.154283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b13b97-fd83-4d37-b77c-e1b70d12f36b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:19.161313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a7f5b-ba29-4652-91f6-a7008b2c84fa None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:19.171102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 306/628] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:19.171102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43c893-c319-4d87-ad9e-29362ce7b902 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.171102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43c893-c319-4d87-ad9e-29362ce7b902 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.171102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43c893-c319-4d87-ad9e-29362ce7b902 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.171102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 307/629] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:19.171578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43c893-c319-4d87-ad9e-29362ce7b902 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:19.197695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67035933-7529-4bbc-9e87-e71e11875aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:19.198431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67035933-7529-4bbc-9e87-e71e11875aed None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.198711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67035933-7529-4bbc-9e87-e71e11875aed None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.198979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67035933-7529-4bbc-9e87-e71e11875aed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.209317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67035933-7529-4bbc-9e87-e71e11875aed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:19.210772 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67035933-7529-4bbc-9e87-e71e11875aed None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-67035933-7529-4bbc-9e87-e71e11875aed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:19.225714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:19.226689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:19.227821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:19.227821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:19.227821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:19.227821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:19.227821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:19.227821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:19.227821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:19.227821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:19.227821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:19.227821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:19.231918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 323/630] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:19.232803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67035933-7529-4bbc-9e87-e71e11875aed None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:19.275902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43c893-c319-4d87-ad9e-29362ce7b902 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:19.307177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853e73f-2085-445d-a01d-23c2e3d66d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:19.307984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853e73f-2085-445d-a01d-23c2e3d66d36 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.308242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853e73f-2085-445d-a01d-23c2e3d66d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.308533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853e73f-2085-445d-a01d-23c2e3d66d36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.319876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853e73f-2085-445d-a01d-23c2e3d66d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b853e73f-2085-445d-a01d-23c2e3d66d36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:19.328236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:19.329541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:19.330411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:19.331632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67035933-7529-4bbc-9e87-e71e11875aed None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:19.334438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 308/631] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:19.336001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6815a5-3ce6-42bf-9189-05cd5d23d526 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.336284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6815a5-3ce6-42bf-9189-05cd5d23d526 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.336564 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6815a5-3ce6-42bf-9189-05cd5d23d526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.355803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9d6815a5-3ce6-42bf-9189-05cd5d23d526 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:19.357591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9d6815a5-3ce6-42bf-9189-05cd5d23d526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:19.374623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b09ca-5b09-4eba-9d7e-cf804a071bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:19.375442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b09ca-5b09-4eba-9d7e-cf804a071bdc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.375787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b09ca-5b09-4eba-9d7e-cf804a071bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.376119 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b09ca-5b09-4eba-9d7e-cf804a071bdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.377057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b09ca-5b09-4eba-9d7e-cf804a071bdc None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=341358254d884b278d7fa19d95c2dc2d) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:19.377428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b09ca-5b09-4eba-9d7e-cf804a071bdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:19.378923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b09ca-5b09-4eba-9d7e-cf804a071bdc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:19.385338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 324/632] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:19 2022] GET /identity/v3/role_assignments?user.id=341358254d884b278d7fa19d95c2dc2d&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:19.386178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b09ca-5b09-4eba-9d7e-cf804a071bdc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:19.449152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 309/633] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:19.450389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6815a5-3ce6-42bf-9189-05cd5d23d526 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:19.518185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b09ca-5b09-4eba-9d7e-cf804a071bdc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:19.561398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3c008-a6aa-4e81-959a-7971025b809a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:19.562078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3c008-a6aa-4e81-959a-7971025b809a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.562357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3c008-a6aa-4e81-959a-7971025b809a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.562622 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3c008-a6aa-4e81-959a-7971025b809a None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.563462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3c008-a6aa-4e81-959a-7971025b809a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:19.563760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3c008-a6aa-4e81-959a-7971025b809a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:19.565253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3c008-a6aa-4e81-959a-7971025b809a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:19.574910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6815a5-3ce6-42bf-9189-05cd5d23d526 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:19.583956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 325/634] 10.222.0.50 () {56 vars in 1135 bytes} [Fri Apr 29 18:30:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:19.584751 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3c008-a6aa-4e81-959a-7971025b809a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:19.612464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e535-6731-408a-84e9-32c11d04b1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:19.613154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e535-6731-408a-84e9-32c11d04b1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.614259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e535-6731-408a-84e9-32c11d04b1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.614528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e535-6731-408a-84e9-32c11d04b1c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.615276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e535-6731-408a-84e9-32c11d04b1c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:19.615575 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e535-6731-408a-84e9-32c11d04b1c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:19.616983 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e535-6731-408a-84e9-32c11d04b1c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:19.626859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 310/635] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:19.628130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7d16f-c6a3-4c45-b351-4419c98842ea None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.628479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7d16f-c6a3-4c45-b351-4419c98842ea None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.628858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7d16f-c6a3-4c45-b351-4419c98842ea None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.635492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 311/636] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:19.635492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7d16f-c6a3-4c45-b351-4419c98842ea None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:19.685738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3c008-a6aa-4e81-959a-7971025b809a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:19.720594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28ee41-20b0-40ac-9a10-233b3696773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:19.721412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28ee41-20b0-40ac-9a10-233b3696773a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.721699 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28ee41-20b0-40ac-9a10-233b3696773a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.721973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28ee41-20b0-40ac-9a10-233b3696773a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.732445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28ee41-20b0-40ac-9a10-233b3696773a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ab28ee41-20b0-40ac-9a10-233b3696773a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:19.740249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:19.741421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:19.742286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 326/637] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:19.743072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28ee41-20b0-40ac-9a10-233b3696773a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:19.756381 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7d16f-c6a3-4c45-b351-4419c98842ea None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:19.788736 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18c3d0-ee7d-4910-ae11-e59f96b26a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:19.789552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18c3d0-ee7d-4910-ae11-e59f96b26a9d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.789904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18c3d0-ee7d-4910-ae11-e59f96b26a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.790180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18c3d0-ee7d-4910-ae11-e59f96b26a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.800391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18c3d0-ee7d-4910-ae11-e59f96b26a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:19.801880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18c3d0-ee7d-4910-ae11-e59f96b26a9d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0f18c3d0-ee7d-4910-ae11-e59f96b26a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:19.818597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:19.819596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:19.826196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:19.826196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:19.826196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:19.826196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:19.826196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 312/638] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:19.826196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55cfe9-3ca1-4008-a034-9e11f08f1ded None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.826196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55cfe9-3ca1-4008-a034-9e11f08f1ded None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.826196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55cfe9-3ca1-4008-a034-9e11f08f1ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.837770 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28ee41-20b0-40ac-9a10-233b3696773a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:19.841819 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6d55cfe9-3ca1-4008-a034-9e11f08f1ded None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:19.843353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6d55cfe9-3ca1-4008-a034-9e11f08f1ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:19.872180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dcf8e-86b7-477e-9021-6938540c0427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:19.873348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dcf8e-86b7-477e-9021-6938540c0427 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:19.873684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dcf8e-86b7-477e-9021-6938540c0427 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:19.874027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dcf8e-86b7-477e-9021-6938540c0427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:19.885319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dcf8e-86b7-477e-9021-6938540c0427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:19.886929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dcf8e-86b7-477e-9021-6938540c0427 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-a61dcf8e-86b7-477e-9021-6938540c0427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:19.903682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:19.904680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:19.905605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:19.905605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:19.905605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:19.905605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:19.905605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 327/639] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:19.906197 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dcf8e-86b7-477e-9021-6938540c0427 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:19.924787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 313/640] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:19.924787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55cfe9-3ca1-4008-a034-9e11f08f1ded None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.005055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dcf8e-86b7-477e-9021-6938540c0427 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.025909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55cfe9-3ca1-4008-a034-9e11f08f1ded None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.036966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864ce8d-38b0-4550-bbb3-85169b283b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.037632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864ce8d-38b0-4550-bbb3-85169b283b54 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.037904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864ce8d-38b0-4550-bbb3-85169b283b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.038183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864ce8d-38b0-4550-bbb3-85169b283b54 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.038916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864ce8d-38b0-4550-bbb3-85169b283b54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:20.039216 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864ce8d-38b0-4550-bbb3-85169b283b54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.040702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864ce8d-38b0-4550-bbb3-85169b283b54 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.048552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 328/641] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:20.053323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864ce8d-38b0-4550-bbb3-85169b283b54 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.060210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c2c33-a20b-4235-b940-eba503547c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.060910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c2c33-a20b-4235-b940-eba503547c96 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.061184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c2c33-a20b-4235-b940-eba503547c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.061452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c2c33-a20b-4235-b940-eba503547c96 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.062174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c2c33-a20b-4235-b940-eba503547c96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:20.062477 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c2c33-a20b-4235-b940-eba503547c96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.063826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c2c33-a20b-4235-b940-eba503547c96 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.078825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 314/642] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:20.079958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c2c33-a20b-4235-b940-eba503547c96 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.160872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864ce8d-38b0-4550-bbb3-85169b283b54 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.179537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c2c33-a20b-4235-b940-eba503547c96 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.198601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b646e9-cca7-4b4d-a750-dab3121a1ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.199271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b646e9-cca7-4b4d-a750-dab3121a1ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.199553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b646e9-cca7-4b4d-a750-dab3121a1ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.199823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b646e9-cca7-4b4d-a750-dab3121a1ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.200561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b646e9-cca7-4b4d-a750-dab3121a1ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:20.200866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b646e9-cca7-4b4d-a750-dab3121a1ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.202830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b646e9-cca7-4b4d-a750-dab3121a1ea8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.220310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 329/643] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:20.220310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b646e9-cca7-4b4d-a750-dab3121a1ea8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.231520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9df572-1bef-4c82-af2f-0a00f0e2a431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.232433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9df572-1bef-4c82-af2f-0a00f0e2a431 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.232810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9df572-1bef-4c82-af2f-0a00f0e2a431 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.233151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9df572-1bef-4c82-af2f-0a00f0e2a431 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.243862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9df572-1bef-4c82-af2f-0a00f0e2a431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-bf9df572-1bef-4c82-af2f-0a00f0e2a431 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:20.251591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:20.252693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 315/644] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:20.253570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9df572-1bef-4c82-af2f-0a00f0e2a431 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.321386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b646e9-cca7-4b4d-a750-dab3121a1ea8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.354895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9bb8f-92b0-4b65-9d6a-279b3b751ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.355705 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9bb8f-92b0-4b65-9d6a-279b3b751ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.355968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9bb8f-92b0-4b65-9d6a-279b3b751ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.356065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9df572-1bef-4c82-af2f-0a00f0e2a431 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.356263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9bb8f-92b0-4b65-9d6a-279b3b751ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.367178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9bb8f-92b0-4b65-9d6a-279b3b751ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.368872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9bb8f-92b0-4b65-9d6a-279b3b751ab7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-67d9bb8f-92b0-4b65-9d6a-279b3b751ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:20.387387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:20.388417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:20.389687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:20.389687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:20.389687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:20.389687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:20.389687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 330/645] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:20.389687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9bb8f-92b0-4b65-9d6a-279b3b751ab7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.398947 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb70a2-d14d-4c5f-ab1f-f5fb1d7dd407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.400233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb70a2-d14d-4c5f-ab1f-f5fb1d7dd407 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.400520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb70a2-d14d-4c5f-ab1f-f5fb1d7dd407 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.400807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb70a2-d14d-4c5f-ab1f-f5fb1d7dd407 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.407475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb70a2-d14d-4c5f-ab1f-f5fb1d7dd407 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.409005 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb70a2-d14d-4c5f-ab1f-f5fb1d7dd407 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7bbb70a2-d14d-4c5f-ab1f-f5fb1d7dd407 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:20.423014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:20.423941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:20.424901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:20.424901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:20.424901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:20.424901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:20.424901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:20.424901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:20.424901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:20.424901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:20.424901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 316/646] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:20.426045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb70a2-d14d-4c5f-ab1f-f5fb1d7dd407 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.495234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9bb8f-92b0-4b65-9d6a-279b3b751ab7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.523617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb70a2-d14d-4c5f-ab1f-f5fb1d7dd407 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.531380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b5895-5356-4867-8858-0f7fc448d17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.532301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b5895-5356-4867-8858-0f7fc448d17b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.532674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b5895-5356-4867-8858-0f7fc448d17b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.533013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b5895-5356-4867-8858-0f7fc448d17b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.540687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b5895-5356-4867-8858-0f7fc448d17b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.542152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b5895-5356-4867-8858-0f7fc448d17b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-1b3b5895-5356-4867-8858-0f7fc448d17b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:20.556151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:20.557151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:20.558098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:20.558098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:20.558098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:20.558098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:20.558098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:20.558098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:20.558098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:20.558098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:20.558098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Apr 29 18:30:20.558098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:20.558848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ab57e-31e6-403d-811b-be60ed1ea17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.558948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 331/647] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:20.558948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b5895-5356-4867-8858-0f7fc448d17b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.560348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab57e-31e6-403d-811b-be60ed1ea17f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.560830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab57e-31e6-403d-811b-be60ed1ea17f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.561271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab57e-31e6-403d-811b-be60ed1ea17f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.562618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ab57e-31e6-403d-811b-be60ed1ea17f None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:20.563752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ab57e-31e6-403d-811b-be60ed1ea17f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.565184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ab57e-31e6-403d-811b-be60ed1ea17f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.578877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 317/648] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:20.578877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ab57e-31e6-403d-811b-be60ed1ea17f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.665621 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b5895-5356-4867-8858-0f7fc448d17b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.673457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ab57e-31e6-403d-811b-be60ed1ea17f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.698054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861055fe-28ca-4fcf-9096-78719b0facb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.698725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861055fe-28ca-4fcf-9096-78719b0facb6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.699002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861055fe-28ca-4fcf-9096-78719b0facb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.699269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861055fe-28ca-4fcf-9096-78719b0facb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.700029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861055fe-28ca-4fcf-9096-78719b0facb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:20.700300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861055fe-28ca-4fcf-9096-78719b0facb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.701868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861055fe-28ca-4fcf-9096-78719b0facb6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.706159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90014b26-1586-45e7-a867-48b33fcec1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.706822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90014b26-1586-45e7-a867-48b33fcec1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.708031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90014b26-1586-45e7-a867-48b33fcec1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.708031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90014b26-1586-45e7-a867-48b33fcec1c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.708215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90014b26-1586-45e7-a867-48b33fcec1c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:20.708618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90014b26-1586-45e7-a867-48b33fcec1c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.710955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90014b26-1586-45e7-a867-48b33fcec1c5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.720283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 332/649] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:20.720283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861055fe-28ca-4fcf-9096-78719b0facb6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.723047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 318/650] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:20.724617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90014b26-1586-45e7-a867-48b33fcec1c5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.814606 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861055fe-28ca-4fcf-9096-78719b0facb6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.818258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90014b26-1586-45e7-a867-48b33fcec1c5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.849360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaff4b9-777c-43a5-8620-eadec9336adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.849969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaff4b9-777c-43a5-8620-eadec9336adc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.850252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaff4b9-777c-43a5-8620-eadec9336adc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.850582 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaff4b9-777c-43a5-8620-eadec9336adc None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.851359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaff4b9-777c-43a5-8620-eadec9336adc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:20.851663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaff4b9-777c-43a5-8620-eadec9336adc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.853265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaff4b9-777c-43a5-8620-eadec9336adc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.853793 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb9d95-ef65-4275-ad44-b1e8a6820053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:20.854590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb9d95-ef65-4275-ad44-b1e8a6820053 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:20.854871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb9d95-ef65-4275-ad44-b1e8a6820053 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:20.855144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb9d95-ef65-4275-ad44-b1e8a6820053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:20.865069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb9d95-ef65-4275-ad44-b1e8a6820053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:20.866515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb9d95-ef65-4275-ad44-b1e8a6820053 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:20.870789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 333/651] 10.222.0.50 () {56 vars in 1137 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:20.871732 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaff4b9-777c-43a5-8620-eadec9336adc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-56cb9d95-ef65-4275-ad44-b1e8a6820053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:20.884839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:20.885951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:20.886858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:20.886858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:20.886858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:20.886858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:20.886858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 319/652] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:20.886858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb9d95-ef65-4275-ad44-b1e8a6820053 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:20.973378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaff4b9-777c-43a5-8620-eadec9336adc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:20.984513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb9d95-ef65-4275-ad44-b1e8a6820053 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:21.006415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cc9c7-d210-4b83-bfcd-892585ab548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:21.007305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cc9c7-d210-4b83-bfcd-892585ab548f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:21.007652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cc9c7-d210-4b83-bfcd-892585ab548f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:21.007991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cc9c7-d210-4b83-bfcd-892585ab548f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:21.015684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cc9c7-d210-4b83-bfcd-892585ab548f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:21.016580 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8acf79-6f08-4935-a6cf-06b936020cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:21.017157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cc9c7-d210-4b83-bfcd-892585ab548f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:21.017523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8acf79-6f08-4935-a6cf-06b936020cb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:21.017749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8acf79-6f08-4935-a6cf-06b936020cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:21.018097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8acf79-6f08-4935-a6cf-06b936020cb3 None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/4cf46333cd034f5dbd0b78503de1f8e1/roles/4243cffaa7424ff99b0e23fc4940bae5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-a58cc9c7-d210-4b83-bfcd-892585ab548f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:21.033576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:21.034567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 334/653] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:21.035518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cc9c7-d210-4b83-bfcd-892585ab548f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:21.039770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8acf79-6f08-4935-a6cf-06b936020cb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=4cf46333cd034f5dbd0b78503de1f8e1, role_id=4243cffaa7424ff99b0e23fc4940bae5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:21.041548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8acf79-6f08-4935-a6cf-06b936020cb3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:21.068855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 320/654] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:30:20 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/4cf46333cd034f5dbd0b78503de1f8e1/roles/4243cffaa7424ff99b0e23fc4940bae5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:30:21.069948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8acf79-6f08-4935-a6cf-06b936020cb3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:21.147126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cc9c7-d210-4b83-bfcd-892585ab548f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:21.188658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ea5df-6385-4386-b093-d095c553db3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:21.189367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ea5df-6385-4386-b093-d095c553db3f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:21.189706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ea5df-6385-4386-b093-d095c553db3f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:21.190042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ea5df-6385-4386-b093-d095c553db3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:21.251545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8acf79-6f08-4935-a6cf-06b936020cb3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:21.271752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ea5df-6385-4386-b093-d095c553db3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:21.273364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ea5df-6385-4386-b093-d095c553db3f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:21.310713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b21cc-32ad-4817-87e7-18a6541de791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:21.311694 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b21cc-32ad-4817-87e7-18a6541de791 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:21.312144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b21cc-32ad-4817-87e7-18a6541de791 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:21.312652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b21cc-32ad-4817-87e7-18a6541de791 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:21.313556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b21cc-32ad-4817-87e7-18a6541de791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:21.314024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b21cc-32ad-4817-87e7-18a6541de791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:21.315565 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b21cc-32ad-4817-87e7-18a6541de791 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:21.327396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 321/655] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:21.334936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b21cc-32ad-4817-87e7-18a6541de791 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:21.384505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 335/656] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:21.385031 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ea5df-6385-4386-b093-d095c553db3f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:21.496524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ea5df-6385-4386-b093-d095c553db3f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:21.499603 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b21cc-32ad-4817-87e7-18a6541de791 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:21.519500 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2f7ec-bb99-426b-8a9e-b892385c9538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:21.520259 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2f7ec-bb99-426b-8a9e-b892385c9538 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:21.520641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2f7ec-bb99-426b-8a9e-b892385c9538 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:21.520976 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2f7ec-bb99-426b-8a9e-b892385c9538 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:21.530441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2f7ec-bb99-426b-8a9e-b892385c9538 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:21.530441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2f7ec-bb99-426b-8a9e-b892385c9538 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-8cd2f7ec-bb99-426b-8a9e-b892385c9538 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:21.545411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:21.547762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:21.547762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:21.547762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:21.547762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:21.547762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:21.547762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:21.547762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:21.547762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:21.547762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Apr 29 18:30:21.547762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:21.560529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 336/657] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:21.560529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2f7ec-bb99-426b-8a9e-b892385c9538 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:21.664518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2f7ec-bb99-426b-8a9e-b892385c9538 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:21.693283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1db29-653e-4771-8758-5af60c95e556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:21.693927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1db29-653e-4771-8758-5af60c95e556 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:21.696538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1db29-653e-4771-8758-5af60c95e556 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:21.696538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1db29-653e-4771-8758-5af60c95e556 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:21.696538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1db29-653e-4771-8758-5af60c95e556 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:21.696538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1db29-653e-4771-8758-5af60c95e556 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:21.702250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1db29-653e-4771-8758-5af60c95e556 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:21.707611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8a8cc-a643-4134-a592-9711e604c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:21.707611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8a8cc-a643-4134-a592-9711e604c1db None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:21.707611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8a8cc-a643-4134-a592-9711e604c1db None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:21.707611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8a8cc-a643-4134-a592-9711e604c1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:21.723378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8a8cc-a643-4134-a592-9711e604c1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:21.726402 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8a8cc-a643-4134-a592-9711e604c1db None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:21.756364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 322/658] 10.222.0.50 () {56 vars in 1137 bytes} [Fri Apr 29 18:30:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:21.756364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1db29-653e-4771-8758-5af60c95e556 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-dbe8a8cc-a643-4134-a592-9711e604c1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:21.764511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:21.765395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:21.766304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:21.766304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:21.766304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:21.766304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:21.771744 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 337/659] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:21.771744 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8a8cc-a643-4134-a592-9711e604c1db None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:21.860647 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1db29-653e-4771-8758-5af60c95e556 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:21.869384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8a8cc-a643-4134-a592-9711e604c1db None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:21.901742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0817cd-1d32-4021-a216-bf75235170f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:21.902503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0817cd-1d32-4021-a216-bf75235170f0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:21.902874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0817cd-1d32-4021-a216-bf75235170f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:21.903215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0817cd-1d32-4021-a216-bf75235170f0 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:21.904116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0817cd-1d32-4021-a216-bf75235170f0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:21.904512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0817cd-1d32-4021-a216-bf75235170f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:21.906644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0817cd-1d32-4021-a216-bf75235170f0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:21.913242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeede06-b249-4f15-b60a-250127999062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:21.914229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeede06-b249-4f15-b60a-250127999062 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:21.914714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeede06-b249-4f15-b60a-250127999062 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:21.915432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeede06-b249-4f15-b60a-250127999062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:21.930884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 323/660] 10.222.0.50 () {56 vars in 1139 bytes} [Fri Apr 29 18:30:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:21.930884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953689f-b458-4670-b775-91692b15e098 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:21.930884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953689f-b458-4670-b775-91692b15e098 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:21.930884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953689f-b458-4670-b775-91692b15e098 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:21.930884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 324/661] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:21.931952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953689f-b458-4670-b775-91692b15e098 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:21.996551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeede06-b249-4f15-b60a-250127999062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:21.998912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeede06-b249-4f15-b60a-250127999062 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:22.052890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953689f-b458-4670-b775-91692b15e098 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:22.095649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb84d5-e998-4e09-861a-090fd981c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:22.096422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb84d5-e998-4e09-861a-090fd981c500 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:22.096809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb84d5-e998-4e09-861a-090fd981c500 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:22.097146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb84d5-e998-4e09-861a-090fd981c500 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:22.097933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb84d5-e998-4e09-861a-090fd981c500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:22.098308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb84d5-e998-4e09-861a-090fd981c500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:22.099750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb84d5-e998-4e09-861a-090fd981c500 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:22.116422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 325/662] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:22.117353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb84d5-e998-4e09-861a-090fd981c500 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:22.134539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 338/663] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:22.135896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bd94a-60cd-494a-98a0-5bfeb8546b81 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:22.136189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bd94a-60cd-494a-98a0-5bfeb8546b81 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:22.136464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bd94a-60cd-494a-98a0-5bfeb8546b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:22.153757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c51bd94a-60cd-494a-98a0-5bfeb8546b81 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:22.154943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c51bd94a-60cd-494a-98a0-5bfeb8546b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:22.221834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb84d5-e998-4e09-861a-090fd981c500 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:22.276819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 339/664] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:22.277678 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51bd94a-60cd-494a-98a0-5bfeb8546b81 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:22.302628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab822e-9399-43fe-b454-abe9bded9479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:22.303415 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab822e-9399-43fe-b454-abe9bded9479 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:22.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab822e-9399-43fe-b454-abe9bded9479 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:22.304040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab822e-9399-43fe-b454-abe9bded9479 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:22.391336 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab822e-9399-43fe-b454-abe9bded9479 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:22.393239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab822e-9399-43fe-b454-abe9bded9479 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:22.423483 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51bd94a-60cd-494a-98a0-5bfeb8546b81 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:22.478639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cc8ca-047c-4bc9-b91b-f1fa64eae7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:22.480184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cc8ca-047c-4bc9-b91b-f1fa64eae7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:22.480818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cc8ca-047c-4bc9-b91b-f1fa64eae7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:22.482805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cc8ca-047c-4bc9-b91b-f1fa64eae7f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:22.495991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4cc8ca-047c-4bc9-b91b-f1fa64eae7f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:22.497464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4cc8ca-047c-4bc9-b91b-f1fa64eae7f5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-cb4cc8ca-047c-4bc9-b91b-f1fa64eae7f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:22.514507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:22.515401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:22.516291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:22.516291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:22.516291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:22.516291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:22.516291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:22.516291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:22.516291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:22.516291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:22.516291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 340/665] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:22.516808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cc8ca-047c-4bc9-b91b-f1fa64eae7f5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:22.562403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 326/666] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:22.562403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab822e-9399-43fe-b454-abe9bded9479 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:22.662537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cc8ca-047c-4bc9-b91b-f1fa64eae7f5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:22.704084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab822e-9399-43fe-b454-abe9bded9479 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:22.719551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c53224-3e19-4c3c-8539-227168603979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:22.720365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c53224-3e19-4c3c-8539-227168603979 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:22.720665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c53224-3e19-4c3c-8539-227168603979 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:22.720934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c53224-3e19-4c3c-8539-227168603979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:22.732257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c53224-3e19-4c3c-8539-227168603979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:22.733525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c53224-3e19-4c3c-8539-227168603979 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-44c53224-3e19-4c3c-8539-227168603979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:22.753145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:22.754254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:22.755373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:22.755373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:22.755373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:22.755373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:22.755723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5860f9-275f-434a-b057-cdd71ec3cd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:22.756370 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 341/667] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:22.757762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c53224-3e19-4c3c-8539-227168603979 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:22.758875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5860f9-275f-434a-b057-cdd71ec3cd51 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:22.759300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5860f9-275f-434a-b057-cdd71ec3cd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:22.759696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5860f9-275f-434a-b057-cdd71ec3cd51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:22.774944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5860f9-275f-434a-b057-cdd71ec3cd51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-fc5860f9-275f-434a-b057-cdd71ec3cd51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:22.783144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:22.784239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:22.785331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:22.792389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 327/668] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:22.793450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5860f9-275f-434a-b057-cdd71ec3cd51 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:22.911486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5860f9-275f-434a-b057-cdd71ec3cd51 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:22.923263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c53224-3e19-4c3c-8539-227168603979 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:22.945792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c85ade-c443-47a8-8994-25ab85daf73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:22.946692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c85ade-c443-47a8-8994-25ab85daf73b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:22.946968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c85ade-c443-47a8-8994-25ab85daf73b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:22.947236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c85ade-c443-47a8-8994-25ab85daf73b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:22.956895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3dd3d-9029-4f24-b012-65f9ba9b7301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:22.957564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3dd3d-9029-4f24-b012-65f9ba9b7301 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:22.957839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3dd3d-9029-4f24-b012-65f9ba9b7301 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:22.958105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3dd3d-9029-4f24-b012-65f9ba9b7301 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:22.958213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c85ade-c443-47a8-8994-25ab85daf73b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:22.958880 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3dd3d-9029-4f24-b012-65f9ba9b7301 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:22.959177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3dd3d-9029-4f24-b012-65f9ba9b7301 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:22.959636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c85ade-c443-47a8-8994-25ab85daf73b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:22.960617 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3dd3d-9029-4f24-b012-65f9ba9b7301 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:22.973663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 342/669] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:22.974362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3dd3d-9029-4f24-b012-65f9ba9b7301 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d0c85ade-c443-47a8-8994-25ab85daf73b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:22.982610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:22.983537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:22.984426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:22.984426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:22.984426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:22.984426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:22.985881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 328/670] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:22.985881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c85ade-c443-47a8-8994-25ab85daf73b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.077560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3dd3d-9029-4f24-b012-65f9ba9b7301 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.083314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c85ade-c443-47a8-8994-25ab85daf73b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.114956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cf2af-fdde-41ad-8653-ec7892848aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.115629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cf2af-fdde-41ad-8653-ec7892848aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.115910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cf2af-fdde-41ad-8653-ec7892848aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.116185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cf2af-fdde-41ad-8653-ec7892848aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.116937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cf2af-fdde-41ad-8653-ec7892848aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:23.117240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cf2af-fdde-41ad-8653-ec7892848aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.117883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12108575-e2cd-4a0b-b055-290c70ba11a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.118555 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12108575-e2cd-4a0b-b055-290c70ba11a9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.118767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cf2af-fdde-41ad-8653-ec7892848aa5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:23.118860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12108575-e2cd-4a0b-b055-290c70ba11a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.119093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12108575-e2cd-4a0b-b055-290c70ba11a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.119827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12108575-e2cd-4a0b-b055-290c70ba11a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:23.120313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12108575-e2cd-4a0b-b055-290c70ba11a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.122162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12108575-e2cd-4a0b-b055-290c70ba11a9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:23.130008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 343/671] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:23.131107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cf2af-fdde-41ad-8653-ec7892848aa5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.135955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 329/672] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:23.136880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12108575-e2cd-4a0b-b055-290c70ba11a9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.231201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cf2af-fdde-41ad-8653-ec7892848aa5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.237097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12108575-e2cd-4a0b-b055-290c70ba11a9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.265181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b62ab6-ee1c-482f-b68a-b94705a35fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.265846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b62ab6-ee1c-482f-b68a-b94705a35fbb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.266131 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b62ab6-ee1c-482f-b68a-b94705a35fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.266401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b62ab6-ee1c-482f-b68a-b94705a35fbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.267266 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b62ab6-ee1c-482f-b68a-b94705a35fbb None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=6552c41f1c654ef5bfdd940e3b807dae) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:23.267520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b62ab6-ee1c-482f-b68a-b94705a35fbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.268970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b62ab6-ee1c-482f-b68a-b94705a35fbb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:23.270592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da506b0-b5af-4d71-af5d-ddd5a2d4442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.271215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da506b0-b5af-4d71-af5d-ddd5a2d4442a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.271783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da506b0-b5af-4d71-af5d-ddd5a2d4442a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.271783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da506b0-b5af-4d71-af5d-ddd5a2d4442a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.272455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da506b0-b5af-4d71-af5d-ddd5a2d4442a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:23.272786 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da506b0-b5af-4d71-af5d-ddd5a2d4442a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.274350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da506b0-b5af-4d71-af5d-ddd5a2d4442a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:23.275363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 344/673] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/role_assignments?user.id=6552c41f1c654ef5bfdd940e3b807dae&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:23.282012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b62ab6-ee1c-482f-b68a-b94705a35fbb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.291751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 330/674] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:23.291751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da506b0-b5af-4d71-af5d-ddd5a2d4442a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.408317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b62ab6-ee1c-482f-b68a-b94705a35fbb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.429940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da506b0-b5af-4d71-af5d-ddd5a2d4442a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.454564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07447063-b333-4d76-97b9-9939b709be5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.456568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07447063-b333-4d76-97b9-9939b709be5a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.457001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07447063-b333-4d76-97b9-9939b709be5a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.457693 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07447063-b333-4d76-97b9-9939b709be5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.467938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42232a1-8e6b-4137-bb69-618112334320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.468821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07447063-b333-4d76-97b9-9939b709be5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.469368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42232a1-8e6b-4137-bb69-618112334320 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.469786 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42232a1-8e6b-4137-bb69-618112334320 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.470149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42232a1-8e6b-4137-bb69-618112334320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.471231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07447063-b333-4d76-97b9-9939b709be5a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:23.481544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42232a1-8e6b-4137-bb69-618112334320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.483129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42232a1-8e6b-4137-bb69-618112334320 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-07447063-b333-4d76-97b9-9939b709be5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:23.491757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:23.492969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:23.494255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:23.494255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:23.494255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:23.494255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:23.494255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:23.494255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:23.494255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:23.494255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:23.494255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 345/675] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:23.495808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07447063-b333-4d76-97b9-9939b709be5a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b42232a1-8e6b-4137-bb69-618112334320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:23.508164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:23.509675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:23.515383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:23.515383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:23.515383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:23.515383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:23.515383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 331/676] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:23.515383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42232a1-8e6b-4137-bb69-618112334320 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.614101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07447063-b333-4d76-97b9-9939b709be5a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.625137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42232a1-8e6b-4137-bb69-618112334320 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.650524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09793da6-e26d-4197-a675-be10c1002f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.651361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09793da6-e26d-4197-a675-be10c1002f1b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.651640 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09793da6-e26d-4197-a675-be10c1002f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.651914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09793da6-e26d-4197-a675-be10c1002f1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.658624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09793da6-e26d-4197-a675-be10c1002f1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.660067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09793da6-e26d-4197-a675-be10c1002f1b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:23.661579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dabc8f-9434-4965-844a-162d08427da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.662458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dabc8f-9434-4965-844a-162d08427da6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.662939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dabc8f-9434-4965-844a-162d08427da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.663386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dabc8f-9434-4965-844a-162d08427da6 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.664340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dabc8f-9434-4965-844a-162d08427da6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:23.664920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dabc8f-9434-4965-844a-162d08427da6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.666519 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dabc8f-9434-4965-844a-162d08427da6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-09793da6-e26d-4197-a675-be10c1002f1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:23.673554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:23.674575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:23.675685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:23.675685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:23.675685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:23.675685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:23.675685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:23.675685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:23.675685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:23.675685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:23.675685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 346/677] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:23.676284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09793da6-e26d-4197-a675-be10c1002f1b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.682848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 332/678] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:23.684445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195aec-6b11-42cb-8a75-3eac9c03e41f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.685004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195aec-6b11-42cb-8a75-3eac9c03e41f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.685355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195aec-6b11-42cb-8a75-3eac9c03e41f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.686921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 333/679] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:23.687725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195aec-6b11-42cb-8a75-3eac9c03e41f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.775739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09793da6-e26d-4197-a675-be10c1002f1b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.795544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195aec-6b11-42cb-8a75-3eac9c03e41f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.815091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e27ca2-9c43-4d2a-bd1b-200d7cc76381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.815768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e27ca2-9c43-4d2a-bd1b-200d7cc76381 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.816047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e27ca2-9c43-4d2a-bd1b-200d7cc76381 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.816320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e27ca2-9c43-4d2a-bd1b-200d7cc76381 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.817098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e27ca2-9c43-4d2a-bd1b-200d7cc76381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:23.817383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e27ca2-9c43-4d2a-bd1b-200d7cc76381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.818752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e27ca2-9c43-4d2a-bd1b-200d7cc76381 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:23.830777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 347/680] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:23.832879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71b26b-eb9b-4576-8355-7c4f23c1144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.833633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71b26b-eb9b-4576-8355-7c4f23c1144d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.833986 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71b26b-eb9b-4576-8355-7c4f23c1144d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.834329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71b26b-eb9b-4576-8355-7c4f23c1144d None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.835198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71b26b-eb9b-4576-8355-7c4f23c1144d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:23.835553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71b26b-eb9b-4576-8355-7c4f23c1144d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.835893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e27ca2-9c43-4d2a-bd1b-200d7cc76381 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.837170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71b26b-eb9b-4576-8355-7c4f23c1144d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:23.852302 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 334/681] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:23.853837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173b799-ecf4-49ff-9995-0a61b58fa846 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.854129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173b799-ecf4-49ff-9995-0a61b58fa846 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.854406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173b799-ecf4-49ff-9995-0a61b58fa846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.870471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b173b799-ecf4-49ff-9995-0a61b58fa846 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:23.871750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b173b799-ecf4-49ff-9995-0a61b58fa846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:23.935141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 335/682] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:23.936368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173b799-ecf4-49ff-9995-0a61b58fa846 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:23.948555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e27ca2-9c43-4d2a-bd1b-200d7cc76381 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:23.984277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0e8a8-9bb7-460b-92e7-3b156d2395dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:23.985331 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0e8a8-9bb7-460b-92e7-3b156d2395dd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:23.985788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0e8a8-9bb7-460b-92e7-3b156d2395dd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:23.986061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0e8a8-9bb7-460b-92e7-3b156d2395dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:23.986885 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0e8a8-9bb7-460b-92e7-3b156d2395dd None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=ae4fe640bb564a8abe872cb1ec74dc7d) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:23.987198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0e8a8-9bb7-460b-92e7-3b156d2395dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:23.988643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0e8a8-9bb7-460b-92e7-3b156d2395dd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:24.004525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 348/683] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/role_assignments?user.id=ae4fe640bb564a8abe872cb1ec74dc7d&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:24.004525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0e8a8-9bb7-460b-92e7-3b156d2395dd None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:24.043960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173b799-ecf4-49ff-9995-0a61b58fa846 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:24.103551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947f11a-b6a2-4743-9c10-67f7733f552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:24.108521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947f11a-b6a2-4743-9c10-67f7733f552e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:24.108521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947f11a-b6a2-4743-9c10-67f7733f552e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:24.108521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947f11a-b6a2-4743-9c10-67f7733f552e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:24.119572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947f11a-b6a2-4743-9c10-67f7733f552e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:24.121807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947f11a-b6a2-4743-9c10-67f7733f552e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:24.134842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0e8a8-9bb7-460b-92e7-3b156d2395dd None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5947f11a-b6a2-4743-9c10-67f7733f552e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:24.146604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:24.148307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:24.150125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:24.150125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:24.150125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:24.150125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:24.150125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:24.150125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:24.150125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:24.150125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:24.150125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:24.150125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:24.156624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 336/684] 10.222.0.50 () {56 vars in 1093 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:24.156624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947f11a-b6a2-4743-9c10-67f7733f552e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:24.189447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a0547-a488-41a3-bf2f-3dc3cca34673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:24.189447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a0547-a488-41a3-bf2f-3dc3cca34673 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:24.189765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a0547-a488-41a3-bf2f-3dc3cca34673 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:24.192207 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a0547-a488-41a3-bf2f-3dc3cca34673 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:24.193016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a0547-a488-41a3-bf2f-3dc3cca34673 None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=07849b2925334763b119111ed52dd34c) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:24.193329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a0547-a488-41a3-bf2f-3dc3cca34673 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:24.194808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a0547-a488-41a3-bf2f-3dc3cca34673 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:24.204725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 349/685] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:23 2022] GET /identity/v3/role_assignments?user.id=07849b2925334763b119111ed52dd34c&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:24.204725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a0547-a488-41a3-bf2f-3dc3cca34673 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:24.317973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947f11a-b6a2-4743-9c10-67f7733f552e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:24.336199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a0547-a488-41a3-bf2f-3dc3cca34673 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:24.360328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa3b3-3214-4141-87a5-c6e1cc745a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:24.361177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa3b3-3214-4141-87a5-c6e1cc745a96 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:24.361453 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa3b3-3214-4141-87a5-c6e1cc745a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:24.361718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa3b3-3214-4141-87a5-c6e1cc745a96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:24.377732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fa3b3-3214-4141-87a5-c6e1cc745a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:24.377966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d4b6d-dc90-4214-8c7b-cb4eb35d6e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:24.378626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d4b6d-dc90-4214-8c7b-cb4eb35d6e08 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:24.378965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d4b6d-dc90-4214-8c7b-cb4eb35d6e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:24.379292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d4b6d-dc90-4214-8c7b-cb4eb35d6e08 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:24.380197 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d4b6d-dc90-4214-8c7b-cb4eb35d6e08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:24.380595 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d4b6d-dc90-4214-8c7b-cb4eb35d6e08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:24.382125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d4b6d-dc90-4214-8c7b-cb4eb35d6e08 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3c4fa3b3-3214-4141-87a5-c6e1cc745a96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:24.387668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:24.388707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:24.389546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 337/686] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:24.397548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa3b3-3214-4141-87a5-c6e1cc745a96 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:24.407278 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 350/687] 10.222.0.50 () {56 vars in 1133 bytes} [Fri Apr 29 18:30:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:24.413831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d4b6d-dc90-4214-8c7b-cb4eb35d6e08 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:24.525794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa3b3-3214-4141-87a5-c6e1cc745a96 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:24.530289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d4b6d-dc90-4214-8c7b-cb4eb35d6e08 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:24.568609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e71774-4065-452e-a79b-36ac525d3fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:24.568609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e71774-4065-452e-a79b-36ac525d3fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:24.568609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e71774-4065-452e-a79b-36ac525d3fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:24.568609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e71774-4065-452e-a79b-36ac525d3fd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:24.569350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e71774-4065-452e-a79b-36ac525d3fd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:24.569350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e71774-4065-452e-a79b-36ac525d3fd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:24.572512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e71774-4065-452e-a79b-36ac525d3fd7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:24.572707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6dc78-a8e4-4981-a345-d7ba17606d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:24.575626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6dc78-a8e4-4981-a345-d7ba17606d54 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:24.575626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6dc78-a8e4-4981-a345-d7ba17606d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:24.575626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6dc78-a8e4-4981-a345-d7ba17606d54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:24.581702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 338/688] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:24.586960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e71774-4065-452e-a79b-36ac525d3fd7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:24.669865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6dc78-a8e4-4981-a345-d7ba17606d54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:24.672095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6dc78-a8e4-4981-a345-d7ba17606d54 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:24.721828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e71774-4065-452e-a79b-36ac525d3fd7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:24.762049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4cfaf-f00b-42fa-b364-86d1b08837a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:24.764355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4cfaf-f00b-42fa-b364-86d1b08837a0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:24.764355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4cfaf-f00b-42fa-b364-86d1b08837a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:24.764355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4cfaf-f00b-42fa-b364-86d1b08837a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:24.780622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4cfaf-f00b-42fa-b364-86d1b08837a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:24.780622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4cfaf-f00b-42fa-b364-86d1b08837a0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:24.781105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 351/689] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:24.782594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6dc78-a8e4-4981-a345-d7ba17606d54 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8dc4cfaf-f00b-42fa-b364-86d1b08837a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:24.798420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:24.799390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:24.800924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:24.800924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:24.800924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:24.800924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:24.800924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 339/690] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:24.806794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4cfaf-f00b-42fa-b364-86d1b08837a0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:24.955322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6dc78-a8e4-4981-a345-d7ba17606d54 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:25.003786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc3250-9eb9-40e8-b8ba-715eb33bc460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:25.005066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc3250-9eb9-40e8-b8ba-715eb33bc460 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:25.005551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc3250-9eb9-40e8-b8ba-715eb33bc460 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:25.006030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc3250-9eb9-40e8-b8ba-715eb33bc460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:25.016966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc3250-9eb9-40e8-b8ba-715eb33bc460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:25.018677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc3250-9eb9-40e8-b8ba-715eb33bc460 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-acfc3250-9eb9-40e8-b8ba-715eb33bc460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:25.036568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:25.037507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:25.038386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:25.038386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:25.038386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:25.038386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:25.044610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 352/691] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:25.046570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc3250-9eb9-40e8-b8ba-715eb33bc460 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:25.054439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4cfaf-f00b-42fa-b364-86d1b08837a0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:25.149385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc3250-9eb9-40e8-b8ba-715eb33bc460 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:25.187405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e4da6-5aa8-48b6-88ca-3a9881c76d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:25.188538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e4da6-5aa8-48b6-88ca-3a9881c76d64 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:25.189049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e4da6-5aa8-48b6-88ca-3a9881c76d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:25.189221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac5132-ccc1-4850-be1d-c924d754ac04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:25.189691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e4da6-5aa8-48b6-88ca-3a9881c76d64 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:25.189870 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac5132-ccc1-4850-be1d-c924d754ac04 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:25.190140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac5132-ccc1-4850-be1d-c924d754ac04 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:25.190409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac5132-ccc1-4850-be1d-c924d754ac04 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:25.191143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac5132-ccc1-4850-be1d-c924d754ac04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:25.191383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e4da6-5aa8-48b6-88ca-3a9881c76d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:25.191493 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac5132-ccc1-4850-be1d-c924d754ac04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:25.192016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e4da6-5aa8-48b6-88ca-3a9881c76d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:25.192832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac5132-ccc1-4850-be1d-c924d754ac04 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:25.193787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e4da6-5aa8-48b6-88ca-3a9881c76d64 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:25.212543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 353/692] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:25.214096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e4da6-5aa8-48b6-88ca-3a9881c76d64 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:25.230908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 340/693] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:25.237492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac5132-ccc1-4850-be1d-c924d754ac04 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:25.318821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e4da6-5aa8-48b6-88ca-3a9881c76d64 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:25.354967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cb0f7-fab8-4848-b3a9-6e71595a15a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:25.356157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cb0f7-fab8-4848-b3a9-6e71595a15a4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:25.356664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cb0f7-fab8-4848-b3a9-6e71595a15a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:25.357139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cb0f7-fab8-4848-b3a9-6e71595a15a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:25.364430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993cb0f7-fab8-4848-b3a9-6e71595a15a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:25.366325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993cb0f7-fab8-4848-b3a9-6e71595a15a4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-993cb0f7-fab8-4848-b3a9-6e71595a15a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:25.381456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:25.382420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:25.383543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:25.383543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:25.383543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:25.383543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:25.383543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:25.383543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:25.383543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:25.383543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:25.388581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 354/694] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:25.391097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cb0f7-fab8-4848-b3a9-6e71595a15a4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:25.494694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cb0f7-fab8-4848-b3a9-6e71595a15a4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:25.510884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac5132-ccc1-4850-be1d-c924d754ac04 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:25.539808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b7e0c-f84d-4577-9839-bd09a3751049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:25.541285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b7e0c-f84d-4577-9839-bd09a3751049 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:25.541766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b7e0c-f84d-4577-9839-bd09a3751049 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:25.542240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b7e0c-f84d-4577-9839-bd09a3751049 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:25.543239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b7e0c-f84d-4577-9839-bd09a3751049 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:25.543749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b7e0c-f84d-4577-9839-bd09a3751049 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:25.547201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b7e0c-f84d-4577-9839-bd09a3751049 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:25.573851 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 355/695] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:25.575757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b7e0c-f84d-4577-9839-bd09a3751049 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:25.606956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a121ca-4369-462c-b502-b607dfd2162b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:25.607801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a121ca-4369-462c-b502-b607dfd2162b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:25.608084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a121ca-4369-462c-b502-b607dfd2162b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:25.608353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a121ca-4369-462c-b502-b607dfd2162b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:25.628656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a121ca-4369-462c-b502-b607dfd2162b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:25.630159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a121ca-4369-462c-b502-b607dfd2162b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-70a121ca-4369-462c-b502-b607dfd2162b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:25.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:25.656069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:25.657469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 341/696] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:25.662097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a121ca-4369-462c-b502-b607dfd2162b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:25.694152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b7e0c-f84d-4577-9839-bd09a3751049 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:25.739875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd8ce7-4724-4661-b137-964494f90355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:25.740979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd8ce7-4724-4661-b137-964494f90355 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:25.741457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd8ce7-4724-4661-b137-964494f90355 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:25.741958 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd8ce7-4724-4661-b137-964494f90355 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:25.743145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd8ce7-4724-4661-b137-964494f90355 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=341358254d884b278d7fa19d95c2dc2d) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:25.743774 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd8ce7-4724-4661-b137-964494f90355 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:25.745580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd8ce7-4724-4661-b137-964494f90355 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:25.756605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 356/697] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:25 2022] GET /identity/v3/role_assignments?user.id=341358254d884b278d7fa19d95c2dc2d&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=e55ef11844a742f5889243f189bd59bc => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:25.758431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51816d2b-d238-42cc-99b9-2f560ce560fb None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:25.758916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51816d2b-d238-42cc-99b9-2f560ce560fb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:25.759406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51816d2b-d238-42cc-99b9-2f560ce560fb None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:25.765840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 357/698] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:25.767812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4859e-bea3-4570-a570-7f07b0f6e97d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:25.768424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4859e-bea3-4570-a570-7f07b0f6e97d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:25.768917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4859e-bea3-4570-a570-7f07b0f6e97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:25.813752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-85b4859e-bea3-4570-a570-7f07b0f6e97d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:25.815377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-85b4859e-bea3-4570-a570-7f07b0f6e97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:25.820354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a121ca-4369-462c-b502-b607dfd2162b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:25.873410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdea75-4107-41f6-bca7-10e2b4d58c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:25.874173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdea75-4107-41f6-bca7-10e2b4d58c47 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:25.874460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdea75-4107-41f6-bca7-10e2b4d58c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:25.874781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdea75-4107-41f6-bca7-10e2b4d58c47 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:25.875671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdea75-4107-41f6-bca7-10e2b4d58c47 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:25.875978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdea75-4107-41f6-bca7-10e2b4d58c47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:25.878050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdea75-4107-41f6-bca7-10e2b4d58c47 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:25.916844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 358/699] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:25.922603 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 342/700] 10.222.0.50 () {56 vars in 1135 bytes} [Fri Apr 29 18:30:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:25.928562 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4859e-bea3-4570-a570-7f07b0f6e97d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:25.955600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdea75-4107-41f6-bca7-10e2b4d58c47 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:26.126243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdea75-4107-41f6-bca7-10e2b4d58c47 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:26.174504 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4859e-bea3-4570-a570-7f07b0f6e97d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:26.186866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b25ed2-1572-4d93-8f3b-70126b43a7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:26.187672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b25ed2-1572-4d93-8f3b-70126b43a7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:26.187948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b25ed2-1572-4d93-8f3b-70126b43a7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:26.188211 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b25ed2-1572-4d93-8f3b-70126b43a7a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:26.204617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b25ed2-1572-4d93-8f3b-70126b43a7a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:26.211358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b25ed2-1572-4d93-8f3b-70126b43a7a5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:26.234415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d85ece-954a-4719-abb2-2fec5b93612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:26.235071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85ece-954a-4719-abb2-2fec5b93612d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:26.235347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85ece-954a-4719-abb2-2fec5b93612d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:26.235613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d85ece-954a-4719-abb2-2fec5b93612d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-35b25ed2-1572-4d93-8f3b-70126b43a7a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:26.246432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:26.247326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:26.248733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:26.248733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:26.248733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:26.248733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:26.248733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 343/701] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:26.255488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b25ed2-1572-4d93-8f3b-70126b43a7a5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:26.410270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b25ed2-1572-4d93-8f3b-70126b43a7a5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:26.449657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d85ece-954a-4719-abb2-2fec5b93612d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:26.451550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d85ece-954a-4719-abb2-2fec5b93612d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:26.466982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd95191-398e-47f2-a418-347360e41a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:26.469504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd95191-398e-47f2-a418-347360e41a24 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:26.470766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd95191-398e-47f2-a418-347360e41a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:26.471238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd95191-398e-47f2-a418-347360e41a24 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:26.472162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd95191-398e-47f2-a418-347360e41a24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:26.473440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd95191-398e-47f2-a418-347360e41a24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:26.475847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd95191-398e-47f2-a418-347360e41a24 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:26.495293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 344/702] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:26.503731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd95191-398e-47f2-a418-347360e41a24 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:26.664426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 359/703] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 737 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Apr 29 18:30:26.670212 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d85ece-954a-4719-abb2-2fec5b93612d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:26.711811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd95191-398e-47f2-a418-347360e41a24 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:26.758113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106d52f-c52c-4e13-b070-72f6a093ec9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:26.758848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106d52f-c52c-4e13-b070-72f6a093ec9f None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:26.759122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106d52f-c52c-4e13-b070-72f6a093ec9f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:26.759579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106d52f-c52c-4e13-b070-72f6a093ec9f None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:26.760288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106d52f-c52c-4e13-b070-72f6a093ec9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:26.761781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106d52f-c52c-4e13-b070-72f6a093ec9f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:26.785958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 345/704] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:30:26 2022] POST /identity/v3/users => generated 254 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:26.787287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38506355-a85a-473c-9d8b-341485e4e80a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:26.787571 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38506355-a85a-473c-9d8b-341485e4e80a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:26.787933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38506355-a85a-473c-9d8b-341485e4e80a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:26.792639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 346/705] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:26.795499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bffb5-4028-4262-8cfa-8551c603cedb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:26.796428 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bffb5-4028-4262-8cfa-8551c603cedb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:26.796795 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bffb5-4028-4262-8cfa-8551c603cedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:26.817636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bd4bffb5-4028-4262-8cfa-8551c603cedb None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:26.819367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bd4bffb5-4028-4262-8cfa-8551c603cedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:26.860532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d85ece-954a-4719-abb2-2fec5b93612d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:26.949534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd23c4-78cd-4f10-9e34-5104e508010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:26.950274 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd23c4-78cd-4f10-9e34-5104e508010b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:26.950550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd23c4-78cd-4f10-9e34-5104e508010b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:26.951493 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd23c4-78cd-4f10-9e34-5104e508010b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:26.975066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd23c4-78cd-4f10-9e34-5104e508010b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:26.976639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd23c4-78cd-4f10-9e34-5104e508010b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:26.982731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 347/706] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-16cd23c4-78cd-4f10-9e34-5104e508010b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:26.998485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:26.999462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:27.000977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:27.000977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:27.000977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:27.000977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:27.000977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 360/707] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:27.005433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bffb5-4028-4262-8cfa-8551c603cedb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:27.018319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd23c4-78cd-4f10-9e34-5104e508010b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:27.137687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bffb5-4028-4262-8cfa-8551c603cedb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:27.141197 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd23c4-78cd-4f10-9e34-5104e508010b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:27.183592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ada763-754a-4303-a72c-d15ccdbff34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:27.184269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada763-754a-4303-a72c-d15ccdbff34c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:27.184590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada763-754a-4303-a72c-d15ccdbff34c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:27.184834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ada763-754a-4303-a72c-d15ccdbff34c None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:27.185559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ada763-754a-4303-a72c-d15ccdbff34c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:27.185852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ada763-754a-4303-a72c-d15ccdbff34c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:27.187219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ada763-754a-4303-a72c-d15ccdbff34c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:27.188426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d8695-eee9-4c1d-9a73-ffa02781cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:27.189561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d8695-eee9-4c1d-9a73-ffa02781cbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:27.189983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d8695-eee9-4c1d-9a73-ffa02781cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:27.190396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d8695-eee9-4c1d-9a73-ffa02781cbb8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:27.191178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d8695-eee9-4c1d-9a73-ffa02781cbb8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:27.192695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d8695-eee9-4c1d-9a73-ffa02781cbb8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:27.199414 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 348/708] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:27.200927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995ab1a-15f2-401f-805e-2e0d705aa4be None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:27.201222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995ab1a-15f2-401f-805e-2e0d705aa4be None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:27.201558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995ab1a-15f2-401f-805e-2e0d705aa4be None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:27.203051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 349/709] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ba1d8695-eee9-4c1d-9a73-ffa02781cbb8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:27.208205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:27.209219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Apr 29 18:30:27.210164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:27.210987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5c13d-2160-4c77-a9ab-e84b821385a8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:27.210987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5c13d-2160-4c77-a9ab-e84b821385a8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:27.210987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5c13d-2160-4c77-a9ab-e84b821385a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:27.213173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 361/710] 10.222.0.50 () {56 vars in 1129 bytes} [Fri Apr 29 18:30:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:27.214032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d8695-eee9-4c1d-9a73-ffa02781cbb8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:27.230330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-48e5c13d-2160-4c77-a9ab-e84b821385a8 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:27.232382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-48e5c13d-2160-4c77-a9ab-e84b821385a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:27.321926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 350/711] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:27.323126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5c13d-2160-4c77-a9ab-e84b821385a8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:27.347262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d8695-eee9-4c1d-9a73-ffa02781cbb8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:27.397868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde8146-836d-448f-ba4c-a55d9e8954a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:27.398720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde8146-836d-448f-ba4c-a55d9e8954a8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:27.399239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde8146-836d-448f-ba4c-a55d9e8954a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:27.399347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde8146-836d-448f-ba4c-a55d9e8954a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:27.408236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde8146-836d-448f-ba4c-a55d9e8954a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:27.410221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde8146-836d-448f-ba4c-a55d9e8954a8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6fde8146-836d-448f-ba4c-a55d9e8954a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:27.427499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:27.428429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:27.429355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:27.429355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:27.429355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:27.429355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:27.429355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:27.429355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:27.429355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:27.429355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:27.432893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 362/712] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:27.434093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde8146-836d-448f-ba4c-a55d9e8954a8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:27.461708 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5c13d-2160-4c77-a9ab-e84b821385a8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:27.497845 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49550618-f135-4fb8-aa2f-d46d1f3f2c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:27.498223 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49550618-f135-4fb8-aa2f-d46d1f3f2c60 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:27.498505 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49550618-f135-4fb8-aa2f-d46d1f3f2c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:27.498937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49550618-f135-4fb8-aa2f-d46d1f3f2c60 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:27.499704 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49550618-f135-4fb8-aa2f-d46d1f3f2c60 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:27.499945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49550618-f135-4fb8-aa2f-d46d1f3f2c60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:27.502276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49550618-f135-4fb8-aa2f-d46d1f3f2c60 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:27.506216 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 351/713] 10.222.0.50 () {56 vars in 1124 bytes} [Fri Apr 29 18:30:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:27.507007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49550618-f135-4fb8-aa2f-d46d1f3f2c60 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:27.542552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde8146-836d-448f-ba4c-a55d9e8954a8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:27.581440 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843419a-3544-4712-a2a0-af94ef6dcb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:27.582611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843419a-3544-4712-a2a0-af94ef6dcb33 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:27.583123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843419a-3544-4712-a2a0-af94ef6dcb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:27.583595 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843419a-3544-4712-a2a0-af94ef6dcb33 None admin] PATH_INFO: `/v3/services/network` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:27.584463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843419a-3544-4712-a2a0-af94ef6dcb33 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:27.586292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843419a-3544-4712-a2a0-af94ef6dcb33 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-a843419a-3544-4712-a2a0-af94ef6dcb33 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:27.598988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:27.600175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Apr 29 18:30:27.601067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:27.603299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49550618-f135-4fb8-aa2f-d46d1f3f2c60 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:27.613415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 363/714] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:27 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:27.614659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843419a-3544-4712-a2a0-af94ef6dcb33 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:27.639125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed549f-afbf-4d18-a3a9-7cc43ca8150a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:27.639796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed549f-afbf-4d18-a3a9-7cc43ca8150a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:27.640075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed549f-afbf-4d18-a3a9-7cc43ca8150a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:27.640346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed549f-afbf-4d18-a3a9-7cc43ca8150a None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:27.641190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed549f-afbf-4d18-a3a9-7cc43ca8150a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:27.641494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed549f-afbf-4d18-a3a9-7cc43ca8150a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:27.642927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed549f-afbf-4d18-a3a9-7cc43ca8150a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:27.655041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 352/715] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:27.655877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed549f-afbf-4d18-a3a9-7cc43ca8150a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:27.731006 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843419a-3544-4712-a2a0-af94ef6dcb33 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:27.761865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed549f-afbf-4d18-a3a9-7cc43ca8150a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:27.771568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cf0d1-61c0-4a5a-b371-97273416154d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:27.772247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cf0d1-61c0-4a5a-b371-97273416154d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:27.772542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cf0d1-61c0-4a5a-b371-97273416154d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:27.772813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cf0d1-61c0-4a5a-b371-97273416154d None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:27.773881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cf0d1-61c0-4a5a-b371-97273416154d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:27.774492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cf0d1-61c0-4a5a-b371-97273416154d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:27.776338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cf0d1-61c0-4a5a-b371-97273416154d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:27.783988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 364/716] 10.222.0.50 () {56 vars in 1124 bytes} [Fri Apr 29 18:30:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:27.784639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cf0d1-61c0-4a5a-b371-97273416154d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:27.815224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a8dc8-d2b9-4ac9-88f0-bb9033b149cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:27.816318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8dc8-d2b9-4ac9-88f0-bb9033b149cd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:27.816867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8dc8-d2b9-4ac9-88f0-bb9033b149cd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:27.817357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8dc8-d2b9-4ac9-88f0-bb9033b149cd None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:27.818295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a8dc8-d2b9-4ac9-88f0-bb9033b149cd None admin] RBAC: Adding query filter params (name=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:27.818814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a8dc8-d2b9-4ac9-88f0-bb9033b149cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:27.820441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a8dc8-d2b9-4ac9-88f0-bb9033b149cd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:27.832276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 353/717] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:30:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:27.847789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a8dc8-d2b9-4ac9-88f0-bb9033b149cd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:27.978974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cf0d1-61c0-4a5a-b371-97273416154d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:28.042482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c96fa1-51e0-480a-855f-91e4647d1ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:28.043223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c96fa1-51e0-480a-855f-91e4647d1ddd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:28.043596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c96fa1-51e0-480a-855f-91e4647d1ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:28.043963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c96fa1-51e0-480a-855f-91e4647d1ddd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:28.044930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c96fa1-51e0-480a-855f-91e4647d1ddd None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=4cf46333cd034f5dbd0b78503de1f8e1) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:28.045397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c96fa1-51e0-480a-855f-91e4647d1ddd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:28.046865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c96fa1-51e0-480a-855f-91e4647d1ddd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:28.051890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a8dc8-d2b9-4ac9-88f0-bb9033b149cd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:28.053307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 365/718] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:27 2022] GET /identity/v3/role_assignments?user.id=4cf46333cd034f5dbd0b78503de1f8e1&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:28.091339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cbaf0-eb06-4943-afdf-6d0af2bf0419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:28.092804 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cbaf0-eb06-4943-afdf-6d0af2bf0419 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:28.093237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cbaf0-eb06-4943-afdf-6d0af2bf0419 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:28.093699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cbaf0-eb06-4943-afdf-6d0af2bf0419 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:28.094592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cbaf0-eb06-4943-afdf-6d0af2bf0419 None admin] RBAC: Adding query filter params (type=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:28.095052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cbaf0-eb06-4943-afdf-6d0af2bf0419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:28.096729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cbaf0-eb06-4943-afdf-6d0af2bf0419 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:28.108661 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 354/719] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:30:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:28.359577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c30f05-1761-4bd9-bad1-d97e529b64de None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:28.360247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c30f05-1761-4bd9-bad1-d97e529b64de None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:28.360751 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c30f05-1761-4bd9-bad1-d97e529b64de None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:28.362569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 366/720] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:28.368579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f2de4-f9fe-4ed5-88fd-66a166de983e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:28.369214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f2de4-f9fe-4ed5-88fd-66a166de983e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:28.369692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f2de4-f9fe-4ed5-88fd-66a166de983e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:28.389639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9b1f2de4-f9fe-4ed5-88fd-66a166de983e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:28.390787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9b1f2de4-f9fe-4ed5-88fd-66a166de983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:28.467763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 355/721] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:28.490728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c30f05-1761-4bd9-bad1-d97e529b64de None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:28.601015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c30f05-1761-4bd9-bad1-d97e529b64de None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:28.636965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e734c67-313e-4017-abcc-a72879827ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:28.637781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e734c67-313e-4017-abcc-a72879827ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:28.638061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e734c67-313e-4017-abcc-a72879827ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:28.638331 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e734c67-313e-4017-abcc-a72879827ea6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:28.649620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e734c67-313e-4017-abcc-a72879827ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-4e734c67-313e-4017-abcc-a72879827ea6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:28.657172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:28.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:28.659394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 367/722] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:28.666475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f2de4-f9fe-4ed5-88fd-66a166de983e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:28.764870 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f2de4-f9fe-4ed5-88fd-66a166de983e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:28.800811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25616e41-f071-46f4-baf5-b1dd381d83cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:28.801490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25616e41-f071-46f4-baf5-b1dd381d83cc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:28.801764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25616e41-f071-46f4-baf5-b1dd381d83cc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:28.804510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25616e41-f071-46f4-baf5-b1dd381d83cc None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:28.804510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25616e41-f071-46f4-baf5-b1dd381d83cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:28.804510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25616e41-f071-46f4-baf5-b1dd381d83cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:28.804510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25616e41-f071-46f4-baf5-b1dd381d83cc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:28.813669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 356/723] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:28.819175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e734c67-313e-4017-abcc-a72879827ea6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:28.985613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e734c67-313e-4017-abcc-a72879827ea6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:29.056603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a89c8-83a0-456c-a2ab-2b100ed6ae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:29.057797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a89c8-83a0-456c-a2ab-2b100ed6ae7e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:29.058300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a89c8-83a0-456c-a2ab-2b100ed6ae7e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:29.058765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a89c8-83a0-456c-a2ab-2b100ed6ae7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:29.219154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a89c8-83a0-456c-a2ab-2b100ed6ae7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:29.220658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a89c8-83a0-456c-a2ab-2b100ed6ae7e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-7c6a89c8-83a0-456c-a2ab-2b100ed6ae7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:29.241660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:29.242639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:29.243716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:29.243716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:29.243716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:29.243716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:29.243716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 368/724] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:29.253628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25616e41-f071-46f4-baf5-b1dd381d83cc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:29.412045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25616e41-f071-46f4-baf5-b1dd381d83cc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:29.459563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ced4be-d8af-4e53-9923-b7c765f88ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:29.460249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ced4be-d8af-4e53-9923-b7c765f88ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:29.460700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ced4be-d8af-4e53-9923-b7c765f88ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:29.460896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ced4be-d8af-4e53-9923-b7c765f88ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:29.461722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ced4be-d8af-4e53-9923-b7c765f88ba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:29.462029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ced4be-d8af-4e53-9923-b7c765f88ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:29.463402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ced4be-d8af-4e53-9923-b7c765f88ba2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:29.482524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 357/725] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:29.489590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a89c8-83a0-456c-a2ab-2b100ed6ae7e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:29.567667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d941aa-1b79-4f06-ad60-22a2bf991ad9 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:29.567861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d941aa-1b79-4f06-ad60-22a2bf991ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:29.568129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d941aa-1b79-4f06-ad60-22a2bf991ad9 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:29.570628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 358/726] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:29.575618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da00bd-9326-49c7-ba36-15ef7bfeef2c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:29.575920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da00bd-9326-49c7-ba36-15ef7bfeef2c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:29.576188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da00bd-9326-49c7-ba36-15ef7bfeef2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:29.630772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e9da00bd-9326-49c7-ba36-15ef7bfeef2c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:29.632242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e9da00bd-9326-49c7-ba36-15ef7bfeef2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:29.640946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a89c8-83a0-456c-a2ab-2b100ed6ae7e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:29.681932 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866b1a4-6c76-4b8f-a538-a7258da8b47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:29.682745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866b1a4-6c76-4b8f-a538-a7258da8b47f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:29.683025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866b1a4-6c76-4b8f-a538-a7258da8b47f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:29.683298 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866b1a4-6c76-4b8f-a538-a7258da8b47f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:29.700125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866b1a4-6c76-4b8f-a538-a7258da8b47f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:29.704531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866b1a4-6c76-4b8f-a538-a7258da8b47f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-3866b1a4-6c76-4b8f-a538-a7258da8b47f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:29.735175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:29.736199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:29.737160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:29.737160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:29.737160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:29.737160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:29.737160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:29.737160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:29.737160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:29.737160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:29.737160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:29.737160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:29.742393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 369/727] 10.222.0.50 () {56 vars in 1093 bytes} [Fri Apr 29 18:30:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:29.750560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866b1a4-6c76-4b8f-a538-a7258da8b47f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:29.777572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 359/728] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:29.811935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da00bd-9326-49c7-ba36-15ef7bfeef2c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:29.922026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866b1a4-6c76-4b8f-a538-a7258da8b47f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:29.988055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40c753-4421-4199-9be3-eaed5626065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:29.990025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40c753-4421-4199-9be3-eaed5626065a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:29.990613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40c753-4421-4199-9be3-eaed5626065a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:29.991615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40c753-4421-4199-9be3-eaed5626065a None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:29.992906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40c753-4421-4199-9be3-eaed5626065a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:29.994905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da00bd-9326-49c7-ba36-15ef7bfeef2c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:29.995376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40c753-4421-4199-9be3-eaed5626065a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:29.997018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40c753-4421-4199-9be3-eaed5626065a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:30.036870 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 370/729] 10.222.0.50 () {56 vars in 1133 bytes} [Fri Apr 29 18:30:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:30.049668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40c753-4421-4199-9be3-eaed5626065a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:30.066300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd5c19-f76f-4963-bba7-49ee0d61a0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:30.067349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd5c19-f76f-4963-bba7-49ee0d61a0bd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:30.068465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd5c19-f76f-4963-bba7-49ee0d61a0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:30.069095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd5c19-f76f-4963-bba7-49ee0d61a0bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:30.091706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd5c19-f76f-4963-bba7-49ee0d61a0bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9cfd5c19-f76f-4963-bba7-49ee0d61a0bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:30.101701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:30.102921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:30.103755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:30.107677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 360/730] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:30.114357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd5c19-f76f-4963-bba7-49ee0d61a0bd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:30.280341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40c753-4421-4199-9be3-eaed5626065a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:30.345947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2be17b-00d1-4f59-9c96-933f1c328546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:30.347096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2be17b-00d1-4f59-9c96-933f1c328546 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:30.347575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2be17b-00d1-4f59-9c96-933f1c328546 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:30.348060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2be17b-00d1-4f59-9c96-933f1c328546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:30.367121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2be17b-00d1-4f59-9c96-933f1c328546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:30.369038 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2be17b-00d1-4f59-9c96-933f1c328546 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6d2be17b-00d1-4f59-9c96-933f1c328546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:30.404910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:30.406005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:30.406942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:30.406942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:30.406942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:30.406942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:30.420225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 371/731] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:30.433790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2be17b-00d1-4f59-9c96-933f1c328546 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:30.548190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd5c19-f76f-4963-bba7-49ee0d61a0bd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:30.630321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d4113-c7a9-4f22-b9ac-69bf8fbbde0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:30.631319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d4113-c7a9-4f22-b9ac-69bf8fbbde0c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:30.631602 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d4113-c7a9-4f22-b9ac-69bf8fbbde0c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:30.631873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d4113-c7a9-4f22-b9ac-69bf8fbbde0c None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:30.632919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d4113-c7a9-4f22-b9ac-69bf8fbbde0c None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:30.633251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d4113-c7a9-4f22-b9ac-69bf8fbbde0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:30.634713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d4113-c7a9-4f22-b9ac-69bf8fbbde0c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:30.635125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2be17b-00d1-4f59-9c96-933f1c328546 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:30.648656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 361/732] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 535 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:30.652157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78aed5-2c02-44c5-b68c-3908c205908e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:30.652447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78aed5-2c02-44c5-b68c-3908c205908e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:30.652760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78aed5-2c02-44c5-b68c-3908c205908e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:30.654176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 362/733] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:30.658171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78aed5-2c02-44c5-b68c-3908c205908e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:30.683352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601537c8-82f2-455d-a038-d77c01f36a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:30.684418 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601537c8-82f2-455d-a038-d77c01f36a53 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:30.684961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601537c8-82f2-455d-a038-d77c01f36a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:30.685469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601537c8-82f2-455d-a038-d77c01f36a53 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:30.686419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601537c8-82f2-455d-a038-d77c01f36a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:30.686917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601537c8-82f2-455d-a038-d77c01f36a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:30.688509 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601537c8-82f2-455d-a038-d77c01f36a53 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:30.700643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 372/734] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:30.702274 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3866a76-19ef-48e5-9220-1c22cdf463dd None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:30.702773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3866a76-19ef-48e5-9220-1c22cdf463dd None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:30.703233 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3866a76-19ef-48e5-9220-1c22cdf463dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:30.721160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e3866a76-19ef-48e5-9220-1c22cdf463dd None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:30.722722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e3866a76-19ef-48e5-9220-1c22cdf463dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:30.767313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78aed5-2c02-44c5-b68c-3908c205908e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:30.791648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 373/735] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:30.793434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3866a76-19ef-48e5-9220-1c22cdf463dd None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:30.808021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b6aa7-320b-480f-a671-d077d3115439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:30.808786 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b6aa7-320b-480f-a671-d077d3115439 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:30.809063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b6aa7-320b-480f-a671-d077d3115439 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:30.809328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b6aa7-320b-480f-a671-d077d3115439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:30.821292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b6aa7-320b-480f-a671-d077d3115439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:30.822826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b6aa7-320b-480f-a671-d077d3115439 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e69b6aa7-320b-480f-a671-d077d3115439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:30.839728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:30.840693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:30.841613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:30.841613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:30.841613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:30.841613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:30.841613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 363/736] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:30.843587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d45db0-01cb-497e-8ef6-f852e2106acf None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:30.843587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d45db0-01cb-497e-8ef6-f852e2106acf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:30.843587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d45db0-01cb-497e-8ef6-f852e2106acf None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:30.848669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 364/737] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:30.848669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d45db0-01cb-497e-8ef6-f852e2106acf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:30.902828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3866a76-19ef-48e5-9220-1c22cdf463dd None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:30.939702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa70c6a-d81a-42c3-bb2b-d4dc575f10b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:30.941294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa70c6a-d81a-42c3-bb2b-d4dc575f10b6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:30.941742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa70c6a-d81a-42c3-bb2b-d4dc575f10b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:30.942171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa70c6a-d81a-42c3-bb2b-d4dc575f10b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:30.949497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa70c6a-d81a-42c3-bb2b-d4dc575f10b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:30.950868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d45db0-01cb-497e-8ef6-f852e2106acf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:30.951499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa70c6a-d81a-42c3-bb2b-d4dc575f10b6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2aa70c6a-d81a-42c3-bb2b-d4dc575f10b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:30.968568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:30.969533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:30.970410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:30.970410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:30.970410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:30.970410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:30.970410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:30.970410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:30.970410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:30.970410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:30.974465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 374/738] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:30.975322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa70c6a-d81a-42c3-bb2b-d4dc575f10b6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:30.984841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afba925-e814-43fc-85c6-d13532df86d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:30.985598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afba925-e814-43fc-85c6-d13532df86d1 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:30.985896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afba925-e814-43fc-85c6-d13532df86d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:30.986146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afba925-e814-43fc-85c6-d13532df86d1 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:30.986860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afba925-e814-43fc-85c6-d13532df86d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:30.988210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afba925-e814-43fc-85c6-d13532df86d1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:31.003839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 365/739] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:30:30 2022] POST /identity/v3/services => generated 251 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:31.005558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469be81-18f4-41cd-902c-9dab2918e204 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.005914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469be81-18f4-41cd-902c-9dab2918e204 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.006261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469be81-18f4-41cd-902c-9dab2918e204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.024530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0469be81-18f4-41cd-902c-9dab2918e204 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:31.025665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0469be81-18f4-41cd-902c-9dab2918e204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:31.103463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa70c6a-d81a-42c3-bb2b-d4dc575f10b6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:31.120528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 366/740] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Apr 29 18:30:31.120528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469be81-18f4-41cd-902c-9dab2918e204 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:31.152291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e438c4-f859-4408-be94-e38bd6a101b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:31.153016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e438c4-f859-4408-be94-e38bd6a101b6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.153300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e438c4-f859-4408-be94-e38bd6a101b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.153572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e438c4-f859-4408-be94-e38bd6a101b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.154306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e438c4-f859-4408-be94-e38bd6a101b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:31.154611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e438c4-f859-4408-be94-e38bd6a101b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:31.156021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e438c4-f859-4408-be94-e38bd6a101b6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:31.176258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 375/741] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:31.176258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c5e56-7c4a-4412-ba3d-3a21489d3900 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.176258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c5e56-7c4a-4412-ba3d-3a21489d3900 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.176258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c5e56-7c4a-4412-ba3d-3a21489d3900 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.176915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 376/742] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:31.178024 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c5e56-7c4a-4412-ba3d-3a21489d3900 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:31.232626 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469be81-18f4-41cd-902c-9dab2918e204 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:31.267310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd085d10-fdc6-46f9-afb2-7619fc3de423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:31.267980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd085d10-fdc6-46f9-afb2-7619fc3de423 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.268290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd085d10-fdc6-46f9-afb2-7619fc3de423 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.268599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd085d10-fdc6-46f9-afb2-7619fc3de423 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.269385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd085d10-fdc6-46f9-afb2-7619fc3de423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:31.269693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd085d10-fdc6-46f9-afb2-7619fc3de423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:31.271121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd085d10-fdc6-46f9-afb2-7619fc3de423 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:31.282987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 367/743] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:31.283714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd085d10-fdc6-46f9-afb2-7619fc3de423 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:31.291481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c5e56-7c4a-4412-ba3d-3a21489d3900 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:31.326659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b650ad-1fa8-42ae-931c-53dd1935ee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:31.327324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b650ad-1fa8-42ae-931c-53dd1935ee79 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.327602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b650ad-1fa8-42ae-931c-53dd1935ee79 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.327873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b650ad-1fa8-42ae-931c-53dd1935ee79 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.328620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b650ad-1fa8-42ae-931c-53dd1935ee79 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:31.329978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b650ad-1fa8-42ae-931c-53dd1935ee79 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:31.346648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 377/744] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:30:31 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:31.347778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7ad4-b169-4240-82fe-b856efdf3e6d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.348095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7ad4-b169-4240-82fe-b856efdf3e6d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.348369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7ad4-b169-4240-82fe-b856efdf3e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.370639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-695d7ad4-b169-4240-82fe-b856efdf3e6d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:31.371672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-695d7ad4-b169-4240-82fe-b856efdf3e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:31.405331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd085d10-fdc6-46f9-afb2-7619fc3de423 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:31.453634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42843a-dc6e-4bbe-8f9a-ab3e364417c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:31.455208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42843a-dc6e-4bbe-8f9a-ab3e364417c2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.455851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42843a-dc6e-4bbe-8f9a-ab3e364417c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.456775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42843a-dc6e-4bbe-8f9a-ab3e364417c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.462382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 378/745] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:31.463287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d7ad4-b169-4240-82fe-b856efdf3e6d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:31.468354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42843a-dc6e-4bbe-8f9a-ab3e364417c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:31.470200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42843a-dc6e-4bbe-8f9a-ab3e364417c2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4c42843a-dc6e-4bbe-8f9a-ab3e364417c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:31.489997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:31.491027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:31.491965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:31.491965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:31.491965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:31.491965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:31.491965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:31.491965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:31.491965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:31.491965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:31.491965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:30:31.491965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:31.495372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 368/746] 10.222.0.50 () {56 vars in 1117 bytes} [Fri Apr 29 18:30:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:31.503503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42843a-dc6e-4bbe-8f9a-ab3e364417c2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:31.620746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42843a-dc6e-4bbe-8f9a-ab3e364417c2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:31.625047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d7ad4-b169-4240-82fe-b856efdf3e6d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:31.652749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fccb79-9896-4724-a7d2-d3dd0dfd5bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:31.653791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fccb79-9896-4724-a7d2-d3dd0dfd5bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.654206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fccb79-9896-4724-a7d2-d3dd0dfd5bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.654633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fccb79-9896-4724-a7d2-d3dd0dfd5bd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.660449 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63700bad-6f5a-4c5e-baad-36491200fa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:31.661611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63700bad-6f5a-4c5e-baad-36491200fa4a None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.661971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63700bad-6f5a-4c5e-baad-36491200fa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.662320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63700bad-6f5a-4c5e-baad-36491200fa4a None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/341358254d884b278d7fa19d95c2dc2d/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.664409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63700bad-6f5a-4c5e-baad-36491200fa4a None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=341358254d884b278d7fa19d95c2dc2d, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:31.666010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fccb79-9896-4724-a7d2-d3dd0dfd5bd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:31.666486 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63700bad-6f5a-4c5e-baad-36491200fa4a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c1fccb79-9896-4724-a7d2-d3dd0dfd5bd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:31.673980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:31.675147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:31.676231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:31.678692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 369/747] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:31.679977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fccb79-9896-4724-a7d2-d3dd0dfd5bd4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:31.684561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 379/748] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:30:31 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/341358254d884b278d7fa19d95c2dc2d/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:30:31.685706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63700bad-6f5a-4c5e-baad-36491200fa4a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:31.783504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fccb79-9896-4724-a7d2-d3dd0dfd5bd4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:31.800302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63700bad-6f5a-4c5e-baad-36491200fa4a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:31.819974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3846f-d387-4264-894f-9e222f0a754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:31.821346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3846f-d387-4264-894f-9e222f0a754c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.821864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3846f-d387-4264-894f-9e222f0a754c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.822311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3846f-d387-4264-894f-9e222f0a754c None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.823397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3846f-d387-4264-894f-9e222f0a754c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:31.823921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3846f-d387-4264-894f-9e222f0a754c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:31.825677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3846f-d387-4264-894f-9e222f0a754c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:31.840673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb2313-aeb0-46af-9585-4b12af1c864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:31.841782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb2313-aeb0-46af-9585-4b12af1c864b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:31.841782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb2313-aeb0-46af-9585-4b12af1c864b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:31.841967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb2313-aeb0-46af-9585-4b12af1c864b None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:31.842978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb2313-aeb0-46af-9585-4b12af1c864b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:31.843280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb2313-aeb0-46af-9585-4b12af1c864b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:31.844800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb2313-aeb0-46af-9585-4b12af1c864b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:31.846980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 370/749] 10.222.0.50 () {56 vars in 1149 bytes} [Fri Apr 29 18:30:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:31.855319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 380/750] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:31.856689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3846f-d387-4264-894f-9e222f0a754c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:31.865848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb2313-aeb0-46af-9585-4b12af1c864b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:31.962883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3846f-d387-4264-894f-9e222f0a754c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:32.001336 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e7f12-c9d4-4f3b-a59b-7ac809cf05ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:32.002375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e7f12-c9d4-4f3b-a59b-7ac809cf05ae None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:32.002847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e7f12-c9d4-4f3b-a59b-7ac809cf05ae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:32.003309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e7f12-c9d4-4f3b-a59b-7ac809cf05ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:32.044307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb2313-aeb0-46af-9585-4b12af1c864b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:32.104130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e7f12-c9d4-4f3b-a59b-7ac809cf05ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:32.105957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e7f12-c9d4-4f3b-a59b-7ac809cf05ae None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:32.107614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33826c89-e76f-4aca-835b-9aad4e57f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:32.108433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33826c89-e76f-4aca-835b-9aad4e57f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:32.108742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33826c89-e76f-4aca-835b-9aad4e57f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:32.109015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33826c89-e76f-4aca-835b-9aad4e57f4c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:32.141420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33826c89-e76f-4aca-835b-9aad4e57f4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:32.142895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33826c89-e76f-4aca-835b-9aad4e57f4c5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-33826c89-e76f-4aca-835b-9aad4e57f4c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:32.174075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:32.175054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:32.175972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:32.175972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:32.175972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:32.175972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:32.175972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 381/751] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:32.182374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33826c89-e76f-4aca-835b-9aad4e57f4c5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:32.219042 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 371/752] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:30:32.229559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e7f12-c9d4-4f3b-a59b-7ac809cf05ae None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:32.343009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e7f12-c9d4-4f3b-a59b-7ac809cf05ae None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:32.370446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33826c89-e76f-4aca-835b-9aad4e57f4c5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:32.394329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818c2b9-7b2f-41fd-ba5c-97a3f74cb48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:32.395534 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818c2b9-7b2f-41fd-ba5c-97a3f74cb48b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:32.396058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818c2b9-7b2f-41fd-ba5c-97a3f74cb48b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:32.396577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818c2b9-7b2f-41fd-ba5c-97a3f74cb48b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:32.405928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892815f-dd1c-47f8-bd04-101cc1c2312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:32.406619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892815f-dd1c-47f8-bd04-101cc1c2312c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:32.406902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892815f-dd1c-47f8-bd04-101cc1c2312c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:32.407178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892815f-dd1c-47f8-bd04-101cc1c2312c None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:32.407924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892815f-dd1c-47f8-bd04-101cc1c2312c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:32.408220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892815f-dd1c-47f8-bd04-101cc1c2312c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:32.410563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892815f-dd1c-47f8-bd04-101cc1c2312c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:32.424306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 382/753] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:32.426594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818c2b9-7b2f-41fd-ba5c-97a3f74cb48b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:32.428260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818c2b9-7b2f-41fd-ba5c-97a3f74cb48b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:32.437562 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892815f-dd1c-47f8-bd04-101cc1c2312c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b818c2b9-7b2f-41fd-ba5c-97a3f74cb48b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:32.447137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:32.448038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:32.449062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:32.449062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:32.449062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:32.449062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:32.451206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 372/754] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:32.456797 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818c2b9-7b2f-41fd-ba5c-97a3f74cb48b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:32.587976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818c2b9-7b2f-41fd-ba5c-97a3f74cb48b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:32.655163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20a9b4-1e2f-4c14-824b-572e95593686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:32.655847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20a9b4-1e2f-4c14-824b-572e95593686 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:32.656121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20a9b4-1e2f-4c14-824b-572e95593686 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:32.656386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20a9b4-1e2f-4c14-824b-572e95593686 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:32.660538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20a9b4-1e2f-4c14-824b-572e95593686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:32.660538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20a9b4-1e2f-4c14-824b-572e95593686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:32.660538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20a9b4-1e2f-4c14-824b-572e95593686 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:32.680523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 373/755] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:32.688039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20a9b4-1e2f-4c14-824b-572e95593686 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:32.704520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892815f-dd1c-47f8-bd04-101cc1c2312c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:32.800391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c3787-48a1-407a-9818-d8c19d024513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:32.804529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c3787-48a1-407a-9818-d8c19d024513 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:32.804529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c3787-48a1-407a-9818-d8c19d024513 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:32.804529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c3787-48a1-407a-9818-d8c19d024513 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:32.820518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c3787-48a1-407a-9818-d8c19d024513 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:32.820518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c3787-48a1-407a-9818-d8c19d024513 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-fb6c3787-48a1-407a-9818-d8c19d024513 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:32.848590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:32.849550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:32.851252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:32.851252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:32.851252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:32.851252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:32.851252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:32.851252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:32.851252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:32.851252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:32.851252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:32.851252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:32.852728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20a9b4-1e2f-4c14-824b-572e95593686 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:32.853203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 383/756] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:32.853203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c3787-48a1-407a-9818-d8c19d024513 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:32.898177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae0285-e5c6-4f48-95a7-89151caf91f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:32.899008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae0285-e5c6-4f48-95a7-89151caf91f3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:32.899289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae0285-e5c6-4f48-95a7-89151caf91f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:32.899556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae0285-e5c6-4f48-95a7-89151caf91f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:32.911043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae0285-e5c6-4f48-95a7-89151caf91f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:32.912721 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae0285-e5c6-4f48-95a7-89151caf91f3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e2ae0285-e5c6-4f48-95a7-89151caf91f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:32.933283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:32.935486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:32.937579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:32.937579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:32.937579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:32.937579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:32.937579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:32.937579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:32.937579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:32.937579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:32.937579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 374/757] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:32.953191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae0285-e5c6-4f48-95a7-89151caf91f3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:33.073072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c3787-48a1-407a-9818-d8c19d024513 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:33.140343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5305d-e418-4b20-93d3-e7504a6eb9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:33.140967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5305d-e418-4b20-93d3-e7504a6eb9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:33.144544 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5305d-e418-4b20-93d3-e7504a6eb9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:33.144544 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5305d-e418-4b20-93d3-e7504a6eb9c8 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:33.144544 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5305d-e418-4b20-93d3-e7504a6eb9c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:33.144544 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5305d-e418-4b20-93d3-e7504a6eb9c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:33.144544 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5305d-e418-4b20-93d3-e7504a6eb9c8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:33.162478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 384/758] 10.222.0.50 () {56 vars in 1135 bytes} [Fri Apr 29 18:30:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:33.168823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5305d-e418-4b20-93d3-e7504a6eb9c8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:33.180022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae0285-e5c6-4f48-95a7-89151caf91f3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:33.273901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f6bd3-6693-4e1b-a8f8-f1e26685fd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:33.274562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f6bd3-6693-4e1b-a8f8-f1e26685fd40 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:33.274845 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f6bd3-6693-4e1b-a8f8-f1e26685fd40 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:33.275120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f6bd3-6693-4e1b-a8f8-f1e26685fd40 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:33.275920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f6bd3-6693-4e1b-a8f8-f1e26685fd40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:33.276232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f6bd3-6693-4e1b-a8f8-f1e26685fd40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:33.277705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f6bd3-6693-4e1b-a8f8-f1e26685fd40 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:33.283066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5305d-e418-4b20-93d3-e7504a6eb9c8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:33.305005 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 375/759] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:33.311806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f6bd3-6693-4e1b-a8f8-f1e26685fd40 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:33.335221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b1889-3f41-476e-8363-5a7beb5a4b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:33.335895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b1889-3f41-476e-8363-5a7beb5a4b2d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:33.336177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b1889-3f41-476e-8363-5a7beb5a4b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:33.336657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b1889-3f41-476e-8363-5a7beb5a4b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:33.479492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f6bd3-6693-4e1b-a8f8-f1e26685fd40 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:33.509650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b1889-3f41-476e-8363-5a7beb5a4b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:33.511503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b1889-3f41-476e-8363-5a7beb5a4b2d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:33.518601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204fa1e-076f-4c30-8afb-8a1b54dedf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:33.522022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204fa1e-076f-4c30-8afb-8a1b54dedf2d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:33.522557 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204fa1e-076f-4c30-8afb-8a1b54dedf2d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:33.523026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204fa1e-076f-4c30-8afb-8a1b54dedf2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:33.524139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204fa1e-076f-4c30-8afb-8a1b54dedf2d None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=b7edbbc857d942d685c124c2c6aac09e) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:33.527206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204fa1e-076f-4c30-8afb-8a1b54dedf2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:33.528903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204fa1e-076f-4c30-8afb-8a1b54dedf2d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:33.540541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 376/760] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:33 2022] GET /identity/v3/role_assignments?user.id=b7edbbc857d942d685c124c2c6aac09e&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:33.626957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c0ced-749f-4f57-87db-aa2bfa0c0a77 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:33.627267 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c0ced-749f-4f57-87db-aa2bfa0c0a77 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:33.627524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c0ced-749f-4f57-87db-aa2bfa0c0a77 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:33.633258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 377/761] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:30:33.635078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb97fa-5e54-4f01-a2f1-e40a53712ccf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:33.635370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb97fa-5e54-4f01-a2f1-e40a53712ccf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:33.635650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb97fa-5e54-4f01-a2f1-e40a53712ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:33.664815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4abb97fa-5e54-4f01-a2f1-e40a53712ccf None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:33.666067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4abb97fa-5e54-4f01-a2f1-e40a53712ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:33.727112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 385/762] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:30:33.729277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b1889-3f41-476e-8363-5a7beb5a4b2d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:33.796705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 378/763] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:33.816515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb97fa-5e54-4f01-a2f1-e40a53712ccf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:33.865638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b1889-3f41-476e-8363-5a7beb5a4b2d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:33.913178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb97fa-5e54-4f01-a2f1-e40a53712ccf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:33.913824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca3618-ce51-43d5-b1b3-e23ca3a9040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:33.916513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca3618-ce51-43d5-b1b3-e23ca3a9040d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:33.916513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca3618-ce51-43d5-b1b3-e23ca3a9040d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:33.916513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca3618-ce51-43d5-b1b3-e23ca3a9040d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:33.946007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca3618-ce51-43d5-b1b3-e23ca3a9040d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:33.947496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca3618-ce51-43d5-b1b3-e23ca3a9040d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:33.951107 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab920b-73bf-402d-ad2c-6060ef43329b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:33.951937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab920b-73bf-402d-ad2c-6060ef43329b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:33.952213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab920b-73bf-402d-ad2c-6060ef43329b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:33.956518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab920b-73bf-402d-ad2c-6060ef43329b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:33.956518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab920b-73bf-402d-ad2c-6060ef43329b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:33.956518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab920b-73bf-402d-ad2c-6060ef43329b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-31ca3618-ce51-43d5-b1b3-e23ca3a9040d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:33.985635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:33.986616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:33.987585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:33.987585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:33.987585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:33.987585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:33.987585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 386/764] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d0ab920b-73bf-402d-ad2c-6060ef43329b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:33.987864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:33.989828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Apr 29 18:30:33.991751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:33.993772 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca3618-ce51-43d5-b1b3-e23ca3a9040d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:33.993914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 379/765] 10.222.0.50 () {56 vars in 1129 bytes} [Fri Apr 29 18:30:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:33.999239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab920b-73bf-402d-ad2c-6060ef43329b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:34.121342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca3618-ce51-43d5-b1b3-e23ca3a9040d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:34.138285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab920b-73bf-402d-ad2c-6060ef43329b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:34.166551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfaf3f-787c-4fe6-bf2c-35724715557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:34.167447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfaf3f-787c-4fe6-bf2c-35724715557c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.167732 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfaf3f-787c-4fe6-bf2c-35724715557c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.168265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfaf3f-787c-4fe6-bf2c-35724715557c None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.169178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfaf3f-787c-4fe6-bf2c-35724715557c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:34.169481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfaf3f-787c-4fe6-bf2c-35724715557c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:34.171255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfaf3f-787c-4fe6-bf2c-35724715557c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:34.172019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd9cf7-6181-4d1d-b5a0-0ecda88ebf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:34.172965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd9cf7-6181-4d1d-b5a0-0ecda88ebf8e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.173457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd9cf7-6181-4d1d-b5a0-0ecda88ebf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.173921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd9cf7-6181-4d1d-b5a0-0ecda88ebf8e None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.174811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd9cf7-6181-4d1d-b5a0-0ecda88ebf8e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:34.175309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd9cf7-6181-4d1d-b5a0-0ecda88ebf8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:34.176932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd9cf7-6181-4d1d-b5a0-0ecda88ebf8e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:34.182266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 380/766] 10.222.0.50 () {56 vars in 1124 bytes} [Fri Apr 29 18:30:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:34.183852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd5fe2-acf4-4ef6-80c3-d77815389b22 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.184338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd5fe2-acf4-4ef6-80c3-d77815389b22 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.184840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd5fe2-acf4-4ef6-80c3-d77815389b22 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.186517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 381/767] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:34.188272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd5fe2-acf4-4ef6-80c3-d77815389b22 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:34.189284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 387/768] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:34.190142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfaf3f-787c-4fe6-bf2c-35724715557c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:34.299021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd5fe2-acf4-4ef6-80c3-d77815389b22 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:34.299229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfaf3f-787c-4fe6-bf2c-35724715557c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:34.330904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e6387-a503-46a6-9817-f016ee775d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:34.331709 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e6387-a503-46a6-9817-f016ee775d77 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.331994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e6387-a503-46a6-9817-f016ee775d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.332177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a189d64-301c-4a93-84fb-14c2603e910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:34.332289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e6387-a503-46a6-9817-f016ee775d77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.333369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a189d64-301c-4a93-84fb-14c2603e910f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.333838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a189d64-301c-4a93-84fb-14c2603e910f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.334291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a189d64-301c-4a93-84fb-14c2603e910f None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.335174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a189d64-301c-4a93-84fb-14c2603e910f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:34.335663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a189d64-301c-4a93-84fb-14c2603e910f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:34.337434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a189d64-301c-4a93-84fb-14c2603e910f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:34.339010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e6387-a503-46a6-9817-f016ee775d77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:34.340439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e6387-a503-46a6-9817-f016ee775d77 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:34.343153 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 382/769] 10.222.0.50 () {56 vars in 1124 bytes} [Fri Apr 29 18:30:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:34.344640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec38e1-a65b-4c2e-b09f-80b9c2c30876 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.345151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec38e1-a65b-4c2e-b09f-80b9c2c30876 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.345612 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec38e1-a65b-4c2e-b09f-80b9c2c30876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-557e6387-a503-46a6-9817-f016ee775d77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:34.356618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:34.357727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:34.359537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:34.359537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:34.359537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:34.359537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:34.359537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:34.359537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:34.359537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:34.359537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:34.359537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 388/770] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:34.360035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f6bbf-023e-4485-9822-0112f9b69447 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.360035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f6bbf-023e-4485-9822-0112f9b69447 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.360406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f6bbf-023e-4485-9822-0112f9b69447 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.364862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 389/771] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:34.366378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f6bbf-023e-4485-9822-0112f9b69447 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:34.369932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-22ec38e1-a65b-4c2e-b09f-80b9c2c30876 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:34.371130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-22ec38e1-a65b-4c2e-b09f-80b9c2c30876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:34.442423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 383/772] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:34.443596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec38e1-a65b-4c2e-b09f-80b9c2c30876 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:34.479669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f6bbf-023e-4485-9822-0112f9b69447 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:34.512899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bbdc1-7063-40fd-bb9c-8524b0c845aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:34.513560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bbdc1-7063-40fd-bb9c-8524b0c845aa None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.513834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bbdc1-7063-40fd-bb9c-8524b0c845aa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.514098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bbdc1-7063-40fd-bb9c-8524b0c845aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.514884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bbdc1-7063-40fd-bb9c-8524b0c845aa None admin] RBAC: Adding query filter params (interface=public, service_id=07bf953c34014e99b35975ed3dddcdae, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:34.515191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bbdc1-7063-40fd-bb9c-8524b0c845aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:34.516650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bbdc1-7063-40fd-bb9c-8524b0c845aa None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:34.523976 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 390/773] 10.222.0.50 () {56 vars in 1251 bytes} [Fri Apr 29 18:30:34 2022] GET /identity/v3/endpoints?service_id=07bf953c34014e99b35975ed3dddcdae&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:34.525210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18345f2-cc17-4fa7-b0ab-060518ce3fae None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.525491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18345f2-cc17-4fa7-b0ab-060518ce3fae None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.525759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18345f2-cc17-4fa7-b0ab-060518ce3fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.541740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f18345f2-cc17-4fa7-b0ab-060518ce3fae None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:34.542900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f18345f2-cc17-4fa7-b0ab-060518ce3fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:34.554945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec38e1-a65b-4c2e-b09f-80b9c2c30876 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:34.592945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab56594-fe82-4991-9f06-92898e08911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:34.593601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab56594-fe82-4991-9f06-92898e08911a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.593875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab56594-fe82-4991-9f06-92898e08911a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.594145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab56594-fe82-4991-9f06-92898e08911a None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.594924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab56594-fe82-4991-9f06-92898e08911a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:34.595223 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab56594-fe82-4991-9f06-92898e08911a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:34.596663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab56594-fe82-4991-9f06-92898e08911a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:34.607867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 384/774] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:34.608698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab56594-fe82-4991-9f06-92898e08911a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:34.629479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 391/775] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:34.630111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18345f2-cc17-4fa7-b0ab-060518ce3fae None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:34.730370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab56594-fe82-4991-9f06-92898e08911a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:34.733594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18345f2-cc17-4fa7-b0ab-060518ce3fae None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:34.762227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea55256-f149-4fb0-8f42-78862b9834c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:34.763016 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea55256-f149-4fb0-8f42-78862b9834c1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.763392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea55256-f149-4fb0-8f42-78862b9834c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.763673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea55256-f149-4fb0-8f42-78862b9834c1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.764303 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea55256-f149-4fb0-8f42-78862b9834c1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:34.765949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea55256-f149-4fb0-8f42-78862b9834c1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:34.766561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea31b1-542a-41f6-86c3-6c1966dbe131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:34.767225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea31b1-542a-41f6-86c3-6c1966dbe131 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.767508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea31b1-542a-41f6-86c3-6c1966dbe131 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.767776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea31b1-542a-41f6-86c3-6c1966dbe131 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.768566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea31b1-542a-41f6-86c3-6c1966dbe131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:34.769924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea31b1-542a-41f6-86c3-6c1966dbe131 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:34.780850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 392/776] 10.222.0.50 () {56 vars in 1087 bytes} [Fri Apr 29 18:30:34 2022] GET /identity/v3/services => generated 772 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:34.780850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea31b1-542a-41f6-86c3-6c1966dbe131 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-dea55256-f149-4fb0-8f42-78862b9834c1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:34.781136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:34.782070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:34.782998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 385/777] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:34 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:34.783787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea55256-f149-4fb0-8f42-78862b9834c1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:34.905316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea55256-f149-4fb0-8f42-78862b9834c1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:34.908543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea31b1-542a-41f6-86c3-6c1966dbe131 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:34.941144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9d0b8-80a7-4a5e-bb81-4b5ef1332ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:34.942726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9d0b8-80a7-4a5e-bb81-4b5ef1332ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.943239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9d0b8-80a7-4a5e-bb81-4b5ef1332ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.943650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9d0b8-80a7-4a5e-bb81-4b5ef1332ed3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.945477 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24619330-0e4c-448d-9d72-bb1c1c1960e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:34.946366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24619330-0e4c-448d-9d72-bb1c1c1960e4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:34.946739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24619330-0e4c-448d-9d72-bb1c1c1960e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:34.947100 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24619330-0e4c-448d-9d72-bb1c1c1960e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:34.948033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24619330-0e4c-448d-9d72-bb1c1c1960e4 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=4cf46333cd034f5dbd0b78503de1f8e1) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:34.948450 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24619330-0e4c-448d-9d72-bb1c1c1960e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:34.950058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24619330-0e4c-448d-9d72-bb1c1c1960e4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:34.964549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 393/778] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:34 2022] GET /identity/v3/role_assignments?user.id=4cf46333cd034f5dbd0b78503de1f8e1&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=e55ef11844a742f5889243f189bd59bc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:34.964549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24619330-0e4c-448d-9d72-bb1c1c1960e4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:34.965394 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9d0b8-80a7-4a5e-bb81-4b5ef1332ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-61d9d0b8-80a7-4a5e-bb81-4b5ef1332ed3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:34.973893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:34.975017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:34.975840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:34.984610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 386/779] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:34.986736 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9d0b8-80a7-4a5e-bb81-4b5ef1332ed3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:35.076747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24619330-0e4c-448d-9d72-bb1c1c1960e4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:35.110770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9d0b8-80a7-4a5e-bb81-4b5ef1332ed3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:35.123840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d7854-eaea-4a57-8c52-28784d622373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:35.123840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d7854-eaea-4a57-8c52-28784d622373 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:35.123840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d7854-eaea-4a57-8c52-28784d622373 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:35.123840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d7854-eaea-4a57-8c52-28784d622373 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:35.124634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d7854-eaea-4a57-8c52-28784d622373 None admin] RBAC: Adding query filter params (name=network) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:35.124634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d7854-eaea-4a57-8c52-28784d622373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:35.124634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d7854-eaea-4a57-8c52-28784d622373 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:35.129392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 394/780] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:30:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:35.140520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d7854-eaea-4a57-8c52-28784d622373 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:35.167992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5cf62-b82f-41c8-8fc7-b572472080c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:35.167992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5cf62-b82f-41c8-8fc7-b572472080c3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:35.167992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5cf62-b82f-41c8-8fc7-b572472080c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:35.167992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5cf62-b82f-41c8-8fc7-b572472080c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:35.168417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5cf62-b82f-41c8-8fc7-b572472080c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:35.168417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5cf62-b82f-41c8-8fc7-b572472080c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:35.169728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5cf62-b82f-41c8-8fc7-b572472080c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:35.179346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 387/781] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:35.188729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5cf62-b82f-41c8-8fc7-b572472080c3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:35.330634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d7854-eaea-4a57-8c52-28784d622373 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:35.396626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ae4dd-063b-44b2-b0dc-383dda27987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:35.397304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae4dd-063b-44b2-b0dc-383dda27987e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:35.397583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae4dd-063b-44b2-b0dc-383dda27987e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:35.397855 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae4dd-063b-44b2-b0dc-383dda27987e None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:35.400077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ae4dd-063b-44b2-b0dc-383dda27987e None admin] RBAC: Adding query filter params (type=network) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:35.400388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ae4dd-063b-44b2-b0dc-383dda27987e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:35.401792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ae4dd-063b-44b2-b0dc-383dda27987e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:35.407191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5cf62-b82f-41c8-8fc7-b572472080c3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:35.417382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 395/782] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:30:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:35.418250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ae4dd-063b-44b2-b0dc-383dda27987e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:35.504013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7bf61-d62f-4ed4-b5ab-b2ab5bc27161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:35.504892 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7bf61-d62f-4ed4-b5ab-b2ab5bc27161 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:35.505177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7bf61-d62f-4ed4-b5ab-b2ab5bc27161 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:35.505449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7bf61-d62f-4ed4-b5ab-b2ab5bc27161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:35.521737 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7bf61-d62f-4ed4-b5ab-b2ab5bc27161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:35.523092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7bf61-d62f-4ed4-b5ab-b2ab5bc27161 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:35.554536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ae4dd-063b-44b2-b0dc-383dda27987e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d9d7bf61-d62f-4ed4-b5ab-b2ab5bc27161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:35.557920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:35.558816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:35.559709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:35.559709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:35.559709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:35.559709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:35.559709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 388/783] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:35.566990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7bf61-d62f-4ed4-b5ab-b2ab5bc27161 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:35.611615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d9c28-d526-4325-950b-3411f0564fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:35.612273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d9c28-d526-4325-950b-3411f0564fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:35.612588 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d9c28-d526-4325-950b-3411f0564fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:35.612860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d9c28-d526-4325-950b-3411f0564fa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:35.613656 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d9c28-d526-4325-950b-3411f0564fa6 None admin] RBAC: Adding query filter params (interface=public, service_id=5196c3c9e93745a793033167b88c494b, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:35.613964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d9c28-d526-4325-950b-3411f0564fa6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:35.615396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d9c28-d526-4325-950b-3411f0564fa6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:35.627826 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 396/784] 10.222.0.50 () {56 vars in 1251 bytes} [Fri Apr 29 18:30:35 2022] GET /identity/v3/endpoints?service_id=5196c3c9e93745a793033167b88c494b&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:35.632879 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d9c28-d526-4325-950b-3411f0564fa6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:35.706839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7bf61-d62f-4ed4-b5ab-b2ab5bc27161 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:35.785655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608bc02-67b8-46c0-8709-b892256ac485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:35.788541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608bc02-67b8-46c0-8709-b892256ac485 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:35.788541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608bc02-67b8-46c0-8709-b892256ac485 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:35.788541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608bc02-67b8-46c0-8709-b892256ac485 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:35.788541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608bc02-67b8-46c0-8709-b892256ac485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:35.788541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608bc02-67b8-46c0-8709-b892256ac485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:35.801955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608bc02-67b8-46c0-8709-b892256ac485 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:35.804051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d9c28-d526-4325-950b-3411f0564fa6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:35.829983 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 389/785] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:35.849068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3659b2-f9b5-4887-b77c-cf261abe8f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:35.849750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3659b2-f9b5-4887-b77c-cf261abe8f00 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:35.850028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3659b2-f9b5-4887-b77c-cf261abe8f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:35.850301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3659b2-f9b5-4887-b77c-cf261abe8f00 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:35.850537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608bc02-67b8-46c0-8709-b892256ac485 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:35.850984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3659b2-f9b5-4887-b77c-cf261abe8f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:35.852667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3659b2-f9b5-4887-b77c-cf261abe8f00 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:35.859255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 397/786] 10.222.0.50 () {56 vars in 1087 bytes} [Fri Apr 29 18:30:35 2022] GET /identity/v3/services => generated 772 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:36.019636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608bc02-67b8-46c0-8709-b892256ac485 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:36.042055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b490d-8917-4293-8394-ec41e39a436b None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:36.042352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b490d-8917-4293-8394-ec41e39a436b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:36.042612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b490d-8917-4293-8394-ec41e39a436b None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:36.044204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 398/787] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:36.050662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20440128-7fe0-40d3-a664-cc1ef2a172d9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:36.050945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20440128-7fe0-40d3-a664-cc1ef2a172d9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:36.051223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20440128-7fe0-40d3-a664-cc1ef2a172d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:36.082239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06e471-2e2c-49e0-94ad-c86e4e240640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:36.083059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06e471-2e2c-49e0-94ad-c86e4e240640 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:36.083339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06e471-2e2c-49e0-94ad-c86e4e240640 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:36.083612 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06e471-2e2c-49e0-94ad-c86e4e240640 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:36.089709 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-20440128-7fe0-40d3-a664-cc1ef2a172d9 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:36.090898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-20440128-7fe0-40d3-a664-cc1ef2a172d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:36.092724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06e471-2e2c-49e0-94ad-c86e4e240640 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:36.094265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06e471-2e2c-49e0-94ad-c86e4e240640 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4b06e471-2e2c-49e0-94ad-c86e4e240640 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:36.110421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:36.111522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:36.112908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 390/788] 10.222.0.50 () {56 vars in 1093 bytes} [Fri Apr 29 18:30:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:36.117912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06e471-2e2c-49e0-94ad-c86e4e240640 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:36.209260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 399/789] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:36.227617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06e471-2e2c-49e0-94ad-c86e4e240640 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:36.237880 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20440128-7fe0-40d3-a664-cc1ef2a172d9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:36.297278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed546108-edfa-49fa-91b3-beb7f48e7f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:36.298302 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed546108-edfa-49fa-91b3-beb7f48e7f01 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:36.298780 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed546108-edfa-49fa-91b3-beb7f48e7f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:36.299245 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed546108-edfa-49fa-91b3-beb7f48e7f01 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:36.300298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed546108-edfa-49fa-91b3-beb7f48e7f01 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:36.300996 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed546108-edfa-49fa-91b3-beb7f48e7f01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:36.302624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed546108-edfa-49fa-91b3-beb7f48e7f01 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:36.322039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 391/790] 10.222.0.50 () {56 vars in 1133 bytes} [Fri Apr 29 18:30:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:36.327843 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed546108-edfa-49fa-91b3-beb7f48e7f01 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:36.426911 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed546108-edfa-49fa-91b3-beb7f48e7f01 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:36.472330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce25224-52ec-42b3-9e46-aa37e4b64be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:36.473289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce25224-52ec-42b3-9e46-aa37e4b64be8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:36.473567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce25224-52ec-42b3-9e46-aa37e4b64be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:36.473996 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce25224-52ec-42b3-9e46-aa37e4b64be8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:36.483302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20440128-7fe0-40d3-a664-cc1ef2a172d9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:36.545607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e360b-e507-4c30-9752-c6cb37250258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:36.546452 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e360b-e507-4c30-9752-c6cb37250258 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:36.546733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e360b-e507-4c30-9752-c6cb37250258 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:36.547000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e360b-e507-4c30-9752-c6cb37250258 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:36.572438 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e360b-e507-4c30-9752-c6cb37250258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-141e360b-e507-4c30-9752-c6cb37250258 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:36.586060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:36.587242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:36.588089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:36.592523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 400/791] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:36.595622 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e360b-e507-4c30-9752-c6cb37250258 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:36.634117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce25224-52ec-42b3-9e46-aa37e4b64be8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:36.635608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce25224-52ec-42b3-9e46-aa37e4b64be8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:36.762270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e360b-e507-4c30-9752-c6cb37250258 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:36.827723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74228aab-2918-42a2-a1e9-e3b0744352e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:36.828332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74228aab-2918-42a2-a1e9-e3b0744352e0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:36.828638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74228aab-2918-42a2-a1e9-e3b0744352e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:36.828912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74228aab-2918-42a2-a1e9-e3b0744352e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:36.829666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74228aab-2918-42a2-a1e9-e3b0744352e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:36.829966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74228aab-2918-42a2-a1e9-e3b0744352e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:36.831355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74228aab-2918-42a2-a1e9-e3b0744352e0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:36.853740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 401/792] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:36.857458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74228aab-2918-42a2-a1e9-e3b0744352e0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:36.877548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 392/793] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:30:36.882922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce25224-52ec-42b3-9e46-aa37e4b64be8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:36.984160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74228aab-2918-42a2-a1e9-e3b0744352e0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:37.018361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fbff0-ce88-44ad-a6a7-9c8dbcbc485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:37.019168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fbff0-ce88-44ad-a6a7-9c8dbcbc485c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.019447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fbff0-ce88-44ad-a6a7-9c8dbcbc485c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.019771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fbff0-ce88-44ad-a6a7-9c8dbcbc485c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.045798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0fbff0-ce88-44ad-a6a7-9c8dbcbc485c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:37.047272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0fbff0-ce88-44ad-a6a7-9c8dbcbc485c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:37.059880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce25224-52ec-42b3-9e46-aa37e4b64be8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ea0fbff0-ce88-44ad-a6a7-9c8dbcbc485c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:37.078918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:37.079854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:37.080784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:37.080784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:37.080784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:37.080784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:37.094836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 402/794] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:37.096522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fbff0-ce88-44ad-a6a7-9c8dbcbc485c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:37.111539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2874b-4322-4fb5-a2b3-3617506d5965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:37.112741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2874b-4322-4fb5-a2b3-3617506d5965 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.113238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2874b-4322-4fb5-a2b3-3617506d5965 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.113702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2874b-4322-4fb5-a2b3-3617506d5965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.126032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2874b-4322-4fb5-a2b3-3617506d5965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:37.128768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2874b-4322-4fb5-a2b3-3617506d5965 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f3e2874b-4322-4fb5-a2b3-3617506d5965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:37.146536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:37.147473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:37.148370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:37.148370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:37.148370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:37.148370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:37.150825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 393/795] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:37.156407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2874b-4322-4fb5-a2b3-3617506d5965 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:37.267110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2874b-4322-4fb5-a2b3-3617506d5965 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:37.301733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee2069-7d64-48b8-b64e-9d38daa2093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:37.303918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fbff0-ce88-44ad-a6a7-9c8dbcbc485c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:37.304824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee2069-7d64-48b8-b64e-9d38daa2093b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.305386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee2069-7d64-48b8-b64e-9d38daa2093b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.305898 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee2069-7d64-48b8-b64e-9d38daa2093b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.306858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee2069-7d64-48b8-b64e-9d38daa2093b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:37.307361 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee2069-7d64-48b8-b64e-9d38daa2093b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:37.308961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee2069-7d64-48b8-b64e-9d38daa2093b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:37.329183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 394/796] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:37.333738 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee2069-7d64-48b8-b64e-9d38daa2093b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:37.349184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b010de-5d4e-4f02-b258-80cc1f86eecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:37.349863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b010de-5d4e-4f02-b258-80cc1f86eecb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.350142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b010de-5d4e-4f02-b258-80cc1f86eecb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.350412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b010de-5d4e-4f02-b258-80cc1f86eecb None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.357928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b010de-5d4e-4f02-b258-80cc1f86eecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:37.357928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b010de-5d4e-4f02-b258-80cc1f86eecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:37.357928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b010de-5d4e-4f02-b258-80cc1f86eecb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:37.372382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 403/797] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:37.384385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b010de-5d4e-4f02-b258-80cc1f86eecb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:37.520371 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee2069-7d64-48b8-b64e-9d38daa2093b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:37.560531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168bad59-078f-4d7e-bcf2-0f400929f486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:37.560531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bad59-078f-4d7e-bcf2-0f400929f486 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.560531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bad59-078f-4d7e-bcf2-0f400929f486 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.560531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bad59-078f-4d7e-bcf2-0f400929f486 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.567336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b010de-5d4e-4f02-b258-80cc1f86eecb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:37.567708 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168bad59-078f-4d7e-bcf2-0f400929f486 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:37.569197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168bad59-078f-4d7e-bcf2-0f400929f486 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-168bad59-078f-4d7e-bcf2-0f400929f486 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:37.589556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:37.590492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:37.591384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:37.591384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:37.591384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:37.591384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:37.591384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:37.591384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:37.591384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:37.591384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:37.598887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 395/798] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:37.617281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbdd39-62fa-41c3-9177-d016c159c53c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.617281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbdd39-62fa-41c3-9177-d016c159c53c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.617281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbdd39-62fa-41c3-9177-d016c159c53c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.617281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 396/799] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:37 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:37.617281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbdd39-62fa-41c3-9177-d016c159c53c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:37.628532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa511583-9485-46a7-9bfb-dec30db3ed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:37.634726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa511583-9485-46a7-9bfb-dec30db3ed4b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.634726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa511583-9485-46a7-9bfb-dec30db3ed4b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.634726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa511583-9485-46a7-9bfb-dec30db3ed4b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.647327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa511583-9485-46a7-9bfb-dec30db3ed4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:37.648987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa511583-9485-46a7-9bfb-dec30db3ed4b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-fa511583-9485-46a7-9bfb-dec30db3ed4b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:37.669777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:37.670778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:37.671727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:37.671727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:37.671727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:37.671727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:37.671727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:37.671727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:37.671727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:37.671727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:37.671727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:30:37.671727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:37.675848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 404/800] 10.222.0.50 () {56 vars in 1117 bytes} [Fri Apr 29 18:30:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:37.677330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625785a1-7e98-485e-ab35-28ab85dbff2e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.677644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625785a1-7e98-485e-ab35-28ab85dbff2e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.677921 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625785a1-7e98-485e-ab35-28ab85dbff2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.699669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-625785a1-7e98-485e-ab35-28ab85dbff2e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:37.700945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-625785a1-7e98-485e-ab35-28ab85dbff2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:37.751459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbdd39-62fa-41c3-9177-d016c159c53c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:37.796646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 405/801] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:37.796646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8def7-cf2c-42d0-bc79-525be2a932f0 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.796646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8def7-cf2c-42d0-bc79-525be2a932f0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.796646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8def7-cf2c-42d0-bc79-525be2a932f0 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.796646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 406/802] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:37.796646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8def7-cf2c-42d0-bc79-525be2a932f0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:37.803426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f82e2-3461-4414-a119-67bf7c05ba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:37.804573 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f82e2-3461-4414-a119-67bf7c05ba16 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.804861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f82e2-3461-4414-a119-67bf7c05ba16 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.805247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f82e2-3461-4414-a119-67bf7c05ba16 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.806196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f82e2-3461-4414-a119-67bf7c05ba16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:37.808556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f82e2-3461-4414-a119-67bf7c05ba16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:37.809058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f82e2-3461-4414-a119-67bf7c05ba16 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:37.831523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 397/803] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:37.831523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15fb85-36cf-4f01-9b7b-dda044b4e642 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.831523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15fb85-36cf-4f01-9b7b-dda044b4e642 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.831523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15fb85-36cf-4f01-9b7b-dda044b4e642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.853101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0b15fb85-36cf-4f01-9b7b-dda044b4e642 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:37.854553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0b15fb85-36cf-4f01-9b7b-dda044b4e642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:37.925846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8def7-cf2c-42d0-bc79-525be2a932f0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:37.936083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 398/804] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:37.937136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15fb85-36cf-4f01-9b7b-dda044b4e642 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:37.975164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09d1e0-f582-476e-8906-1e2f041563e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:37.975851 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09d1e0-f582-476e-8906-1e2f041563e2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:37.976212 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09d1e0-f582-476e-8906-1e2f041563e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:37.976484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09d1e0-f582-476e-8906-1e2f041563e2 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:37.977557 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09d1e0-f582-476e-8906-1e2f041563e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:37.977941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09d1e0-f582-476e-8906-1e2f041563e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:37.979389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09d1e0-f582-476e-8906-1e2f041563e2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:37.999780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 407/805] 10.222.0.50 () {56 vars in 1149 bytes} [Fri Apr 29 18:30:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:38.000852 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09d1e0-f582-476e-8906-1e2f041563e2 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:38.074830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15fb85-36cf-4f01-9b7b-dda044b4e642 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:38.114980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ef64b-cb20-4494-850b-27eadabb463e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:38.116338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ef64b-cb20-4494-850b-27eadabb463e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:38.116835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ef64b-cb20-4494-850b-27eadabb463e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:38.117634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ef64b-cb20-4494-850b-27eadabb463e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:38.120611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09d1e0-f582-476e-8906-1e2f041563e2 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:38.131380 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ef64b-cb20-4494-850b-27eadabb463e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-820ef64b-cb20-4494-850b-27eadabb463e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:38.142465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:38.143429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:38.144271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:38.146480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 399/806] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:38.147020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ef64b-cb20-4494-850b-27eadabb463e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:38.162788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb684ee9-9a6e-47e8-aa47-512b4bd38309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:38.163821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb684ee9-9a6e-47e8-aa47-512b4bd38309 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:38.164198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb684ee9-9a6e-47e8-aa47-512b4bd38309 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:38.164986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb684ee9-9a6e-47e8-aa47-512b4bd38309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:38.165743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb684ee9-9a6e-47e8-aa47-512b4bd38309 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=341358254d884b278d7fa19d95c2dc2d) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:38.166094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb684ee9-9a6e-47e8-aa47-512b4bd38309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:38.167843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb684ee9-9a6e-47e8-aa47-512b4bd38309 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:38.180654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 408/807] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:37 2022] GET /identity/v3/role_assignments?user.id=341358254d884b278d7fa19d95c2dc2d&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=e55ef11844a742f5889243f189bd59bc => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:38.180654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb684ee9-9a6e-47e8-aa47-512b4bd38309 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:38.285471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ef64b-cb20-4494-850b-27eadabb463e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:38.315505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb684ee9-9a6e-47e8-aa47-512b4bd38309 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:38.337992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81674634-a790-4bd0-80fb-070cfd2514d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:38.339008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81674634-a790-4bd0-80fb-070cfd2514d7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:38.339597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81674634-a790-4bd0-80fb-070cfd2514d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:38.340146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81674634-a790-4bd0-80fb-070cfd2514d7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:38.341609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81674634-a790-4bd0-80fb-070cfd2514d7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:38.343746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81674634-a790-4bd0-80fb-070cfd2514d7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-81674634-a790-4bd0-80fb-070cfd2514d7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:38.362055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:38.363086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Apr 29 18:30:38.364024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:38.369836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 400/808] 10.222.0.50 () {56 vars in 1129 bytes} [Fri Apr 29 18:30:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:38.369836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81674634-a790-4bd0-80fb-070cfd2514d7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:38.386067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec85894-6795-4196-83ea-a09177b12fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:38.387329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec85894-6795-4196-83ea-a09177b12fbf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:38.387808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec85894-6795-4196-83ea-a09177b12fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:38.388268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec85894-6795-4196-83ea-a09177b12fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:38.409454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec85894-6795-4196-83ea-a09177b12fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:38.410915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec85894-6795-4196-83ea-a09177b12fbf None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-bec85894-6795-4196-83ea-a09177b12fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:38.437946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:38.438910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:38.439844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:38.439844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:38.439844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:38.439844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:38.440580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 409/809] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:38.441545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec85894-6795-4196-83ea-a09177b12fbf None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:38.541161 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81674634-a790-4bd0-80fb-070cfd2514d7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:38.587644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec85894-6795-4196-83ea-a09177b12fbf None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:38.592667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4796c8f-3410-4505-8ac9-81cfefc7c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:38.593348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4796c8f-3410-4505-8ac9-81cfefc7c22f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:38.593626 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4796c8f-3410-4505-8ac9-81cfefc7c22f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:38.593892 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4796c8f-3410-4505-8ac9-81cfefc7c22f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:38.594636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4796c8f-3410-4505-8ac9-81cfefc7c22f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:38.594937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4796c8f-3410-4505-8ac9-81cfefc7c22f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:38.596315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4796c8f-3410-4505-8ac9-81cfefc7c22f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:38.611427 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 401/810] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:38.612637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4796c8f-3410-4505-8ac9-81cfefc7c22f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:38.638003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f7daa-431f-400b-bf12-1b0b7f692b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:38.639125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f7daa-431f-400b-bf12-1b0b7f692b85 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:38.639503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f7daa-431f-400b-bf12-1b0b7f692b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:38.639870 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f7daa-431f-400b-bf12-1b0b7f692b85 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:38.640998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f7daa-431f-400b-bf12-1b0b7f692b85 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:38.641837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f7daa-431f-400b-bf12-1b0b7f692b85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:38.643836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f7daa-431f-400b-bf12-1b0b7f692b85 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:38.656548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 410/811] 10.222.0.50 () {56 vars in 1124 bytes} [Fri Apr 29 18:30:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:38.656548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f7daa-431f-400b-bf12-1b0b7f692b85 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:38.777469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4796c8f-3410-4505-8ac9-81cfefc7c22f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:38.796313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f7daa-431f-400b-bf12-1b0b7f692b85 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:38.817064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398b06c-b7b0-4b4d-86a3-e8abd0752b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:38.817825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398b06c-b7b0-4b4d-86a3-e8abd0752b49 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:38.818216 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398b06c-b7b0-4b4d-86a3-e8abd0752b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:38.818548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398b06c-b7b0-4b4d-86a3-e8abd0752b49 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:38.819335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398b06c-b7b0-4b4d-86a3-e8abd0752b49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:38.821350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398b06c-b7b0-4b4d-86a3-e8abd0752b49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:38.821898 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398b06c-b7b0-4b4d-86a3-e8abd0752b49 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:38.834187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b8885-3e8a-4819-a050-b379443ec664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:38.834992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b8885-3e8a-4819-a050-b379443ec664 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:38.835273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b8885-3e8a-4819-a050-b379443ec664 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:38.835542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b8885-3e8a-4819-a050-b379443ec664 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:38.840598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 402/812] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:38.841448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398b06c-b7b0-4b4d-86a3-e8abd0752b49 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:38.853245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b8885-3e8a-4819-a050-b379443ec664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:38.854712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b8885-3e8a-4819-a050-b379443ec664 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-c75b8885-3e8a-4819-a050-b379443ec664 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:38.874633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:38.875592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:38.876472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:38.876472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:38.876472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:38.876472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:38.876472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 411/813] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:38.877337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c9c88-e76c-4680-9f24-887d5651dee2 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:38.877337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c9c88-e76c-4680-9f24-887d5651dee2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:38.877573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c9c88-e76c-4680-9f24-887d5651dee2 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:38.883301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 412/814] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:38.883301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c9c88-e76c-4680-9f24-887d5651dee2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:38.976306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398b06c-b7b0-4b4d-86a3-e8abd0752b49 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:39.003057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c9c88-e76c-4680-9f24-887d5651dee2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:39.020800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be731ab-1d4f-45b9-a713-277cd3c074f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:39.021618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be731ab-1d4f-45b9-a713-277cd3c074f2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:39.021903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be731ab-1d4f-45b9-a713-277cd3c074f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:39.022175 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be731ab-1d4f-45b9-a713-277cd3c074f2 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:39.022904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be731ab-1d4f-45b9-a713-277cd3c074f2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:39.023230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be731ab-1d4f-45b9-a713-277cd3c074f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:39.025009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be731ab-1d4f-45b9-a713-277cd3c074f2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:39.035948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 403/815] 10.222.0.50 () {56 vars in 1124 bytes} [Fri Apr 29 18:30:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:39.036831 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be731ab-1d4f-45b9-a713-277cd3c074f2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:39.042226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cbcda-4a9b-45f7-96f6-f7bb803e69f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:39.043029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cbcda-4a9b-45f7-96f6-f7bb803e69f3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:39.043308 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cbcda-4a9b-45f7-96f6-f7bb803e69f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:39.043572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cbcda-4a9b-45f7-96f6-f7bb803e69f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:39.050875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cbcda-4a9b-45f7-96f6-f7bb803e69f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:39.052669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cbcda-4a9b-45f7-96f6-f7bb803e69f3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-725cbcda-4a9b-45f7-96f6-f7bb803e69f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:39.070691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:39.071637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:39.072566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:39.072566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:39.072566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:39.072566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:39.072566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:39.072566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:39.072566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:39.072566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:39.077195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 413/816] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:39.080264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddfda0-4ceb-490b-b956-84982f72dd19 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:39.080264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddfda0-4ceb-490b-b956-84982f72dd19 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:39.080264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddfda0-4ceb-490b-b956-84982f72dd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:39.097291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e0ddfda0-4ceb-490b-b956-84982f72dd19 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:39.098475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e0ddfda0-4ceb-490b-b956-84982f72dd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:39.149525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be731ab-1d4f-45b9-a713-277cd3c074f2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:39.177121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 414/817] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:39.184665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddfda0-4ceb-490b-b956-84982f72dd19 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:39.206531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528263d-22c3-4f48-a25d-a4ed125d1c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:39.207410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528263d-22c3-4f48-a25d-a4ed125d1c0a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:39.207984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528263d-22c3-4f48-a25d-a4ed125d1c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:39.208278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528263d-22c3-4f48-a25d-a4ed125d1c0a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:39.209354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528263d-22c3-4f48-a25d-a4ed125d1c0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:39.211299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528263d-22c3-4f48-a25d-a4ed125d1c0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:39.211487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528263d-22c3-4f48-a25d-a4ed125d1c0a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:39.228814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 404/818] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:39.229046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528263d-22c3-4f48-a25d-a4ed125d1c0a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:39.295733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddfda0-4ceb-490b-b956-84982f72dd19 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:39.343604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479dcf3-226d-4358-86e7-48e208994ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:39.345049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479dcf3-226d-4358-86e7-48e208994ac3 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:39.345492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479dcf3-226d-4358-86e7-48e208994ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:39.345891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479dcf3-226d-4358-86e7-48e208994ac3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:39.346775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479dcf3-226d-4358-86e7-48e208994ac3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:39.348288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479dcf3-226d-4358-86e7-48e208994ac3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:39.352614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528263d-22c3-4f48-a25d-a4ed125d1c0a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:39.383690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 415/819] 10.222.0.50 () {60 vars in 1144 bytes} [Fri Apr 29 18:30:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:39.383690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479dcf3-226d-4358-86e7-48e208994ac3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:39.408615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50d6e9-61bb-4eab-b0bc-3974fc24a833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:39.408615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50d6e9-61bb-4eab-b0bc-3974fc24a833 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:39.408615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50d6e9-61bb-4eab-b0bc-3974fc24a833 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:39.408615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50d6e9-61bb-4eab-b0bc-3974fc24a833 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:39.409017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50d6e9-61bb-4eab-b0bc-3974fc24a833 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:39.409017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50d6e9-61bb-4eab-b0bc-3974fc24a833 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:39.409017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50d6e9-61bb-4eab-b0bc-3974fc24a833 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:39.436520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 405/820] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:39.436520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50d6e9-61bb-4eab-b0bc-3974fc24a833 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:39.547940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479dcf3-226d-4358-86e7-48e208994ac3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:39.611369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b63b7-2bdb-4c22-924a-6f78ef89189d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:39.612186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b63b7-2bdb-4c22-924a-6f78ef89189d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:39.612470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b63b7-2bdb-4c22-924a-6f78ef89189d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:39.612779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b63b7-2bdb-4c22-924a-6f78ef89189d None admin] PATH_INFO: `/v3/services/network` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:39.615407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b63b7-2bdb-4c22-924a-6f78ef89189d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:39.615711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50d6e9-61bb-4eab-b0bc-3974fc24a833 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:39.616844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b63b7-2bdb-4c22-924a-6f78ef89189d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-883b63b7-2bdb-4c22-924a-6f78ef89189d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:39.636330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:39.637659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Apr 29 18:30:39.638726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:39.644267 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 416/821] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:39 2022] GET /identity/v3/services/network => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:39.645169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b63b7-2bdb-4c22-924a-6f78ef89189d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:39.670902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4ddd6-2677-421c-9843-8041ba874f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:39.671106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4ddd6-2677-421c-9843-8041ba874f4c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:39.671271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4ddd6-2677-421c-9843-8041ba874f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:39.671541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4ddd6-2677-421c-9843-8041ba874f4c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:39.686389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4ddd6-2677-421c-9843-8041ba874f4c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:39.687999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4ddd6-2677-421c-9843-8041ba874f4c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c4e4ddd6-2677-421c-9843-8041ba874f4c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:39.703644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:39.704709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:39.706074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 406/822] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:39.710981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4ddd6-2677-421c-9843-8041ba874f4c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:39.776134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b63b7-2bdb-4c22-924a-6f78ef89189d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:39.828115 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca60dbf-0840-43b9-9e41-5644cad09eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:39.829925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca60dbf-0840-43b9-9e41-5644cad09eaa None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:39.831279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca60dbf-0840-43b9-9e41-5644cad09eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:39.833812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca60dbf-0840-43b9-9e41-5644cad09eaa None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/b7edbbc857d942d685c124c2c6aac09e/roles/4243cffaa7424ff99b0e23fc4940bae5` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:39.834621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4ddd6-2677-421c-9843-8041ba874f4c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:39.863057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca60dbf-0840-43b9-9e41-5644cad09eaa None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=b7edbbc857d942d685c124c2c6aac09e, role_id=4243cffaa7424ff99b0e23fc4940bae5)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:39.864896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca60dbf-0840-43b9-9e41-5644cad09eaa None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:39.893971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 417/823] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:30:39 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/b7edbbc857d942d685c124c2c6aac09e/roles/4243cffaa7424ff99b0e23fc4940bae5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:30:39.893971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca60dbf-0840-43b9-9e41-5644cad09eaa None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:39.896293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc2629-3f5f-4399-a174-c663aaeb02f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:39.899213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2629-3f5f-4399-a174-c663aaeb02f4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:39.899536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2629-3f5f-4399-a174-c663aaeb02f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:39.899904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2629-3f5f-4399-a174-c663aaeb02f4 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:39.907803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc2629-3f5f-4399-a174-c663aaeb02f4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:39.907803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc2629-3f5f-4399-a174-c663aaeb02f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:39.909344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc2629-3f5f-4399-a174-c663aaeb02f4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:39.925973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 407/824] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:30:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:39.943160 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc2629-3f5f-4399-a174-c663aaeb02f4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:40.101127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca60dbf-0840-43b9-9e41-5644cad09eaa None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:40.159264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd39915-bea3-46cc-aca0-2f6a5fc3553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:40.159264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39915-bea3-46cc-aca0-2f6a5fc3553c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:40.160821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39915-bea3-46cc-aca0-2f6a5fc3553c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:40.160821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39915-bea3-46cc-aca0-2f6a5fc3553c None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:40.168518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39915-bea3-46cc-aca0-2f6a5fc3553c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:40.168518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39915-bea3-46cc-aca0-2f6a5fc3553c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:40.168518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39915-bea3-46cc-aca0-2f6a5fc3553c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:40.186800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 418/825] 10.222.0.50 () {56 vars in 1135 bytes} [Fri Apr 29 18:30:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:40.193150 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd39915-bea3-46cc-aca0-2f6a5fc3553c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:40.236532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc2629-3f5f-4399-a174-c663aaeb02f4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:40.280338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10511bf2-5480-412c-bd49-50a65a297684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:40.280338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10511bf2-5480-412c-bd49-50a65a297684 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:40.280338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10511bf2-5480-412c-bd49-50a65a297684 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:40.280338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10511bf2-5480-412c-bd49-50a65a297684 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:40.280774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10511bf2-5480-412c-bd49-50a65a297684 None admin] RBAC: Adding query filter params (type=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:40.280774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10511bf2-5480-412c-bd49-50a65a297684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:40.280774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10511bf2-5480-412c-bd49-50a65a297684 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:40.294829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 408/826] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:30:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:40.307073 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10511bf2-5480-412c-bd49-50a65a297684 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:40.405588 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd39915-bea3-46cc-aca0-2f6a5fc3553c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:40.441891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10511bf2-5480-412c-bd49-50a65a297684 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:40.478401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbe251-3a50-4b9f-a33c-664e47b63eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:40.479467 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbe251-3a50-4b9f-a33c-664e47b63eff None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:40.480022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbe251-3a50-4b9f-a33c-664e47b63eff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:40.480517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbe251-3a50-4b9f-a33c-664e47b63eff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:40.481206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd280723-294e-4d54-bb9a-a7cb685e07fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:40.481697 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbe251-3a50-4b9f-a33c-664e47b63eff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:40.482027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd280723-294e-4d54-bb9a-a7cb685e07fe None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:40.482313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd280723-294e-4d54-bb9a-a7cb685e07fe None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:40.482587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd280723-294e-4d54-bb9a-a7cb685e07fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:40.484006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbe251-3a50-4b9f-a33c-664e47b63eff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:40.514539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 409/827] 10.222.0.50 () {60 vars in 1144 bytes} [Fri Apr 29 18:30:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:40.526742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd280723-294e-4d54-bb9a-a7cb685e07fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:40.527086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd280723-294e-4d54-bb9a-a7cb685e07fe None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-fd280723-294e-4d54-bb9a-a7cb685e07fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:40.568834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:40.569815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:40.571223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:40.571223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:40.571223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:40.571223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:40.577051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 419/828] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:40.583170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbe251-3a50-4b9f-a33c-664e47b63eff None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:40.805149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c478-b267-4ea3-a8bb-8c6e1f9bf7fd None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:40.805430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c478-b267-4ea3-a8bb-8c6e1f9bf7fd None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:40.805698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c478-b267-4ea3-a8bb-8c6e1f9bf7fd None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:40.807497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 420/829] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:40.808767 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbe251-3a50-4b9f-a33c-664e47b63eff None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:40.812867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c34ca-6e59-4d72-95b5-8978a570215f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:40.813155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c34ca-6e59-4d72-95b5-8978a570215f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:40.813429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c34ca-6e59-4d72-95b5-8978a570215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:40.849753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3b9c34ca-6e59-4d72-95b5-8978a570215f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:40.851324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3b9c34ca-6e59-4d72-95b5-8978a570215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:40.871504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62b76d-ee79-4b8c-8473-4509fa56c96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:40.872174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b76d-ee79-4b8c-8473-4509fa56c96a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:40.872460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b76d-ee79-4b8c-8473-4509fa56c96a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:40.872760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b76d-ee79-4b8c-8473-4509fa56c96a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:40.874086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b76d-ee79-4b8c-8473-4509fa56c96a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:40.874086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b76d-ee79-4b8c-8473-4509fa56c96a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:40.875163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b76d-ee79-4b8c-8473-4509fa56c96a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:40.896844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 410/830] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:40.900937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62b76d-ee79-4b8c-8473-4509fa56c96a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:40.979942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 421/831] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:41.006211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c34ca-6e59-4d72-95b5-8978a570215f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:41.023673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62b76d-ee79-4b8c-8473-4509fa56c96a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:41.063433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c458863-3c1d-48fd-9367-f7c5a4bb5f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:41.068324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c458863-3c1d-48fd-9367-f7c5a4bb5f30 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:41.068450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c458863-3c1d-48fd-9367-f7c5a4bb5f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:41.068724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c458863-3c1d-48fd-9367-f7c5a4bb5f30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:41.077497 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c458863-3c1d-48fd-9367-f7c5a4bb5f30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:41.078938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c458863-3c1d-48fd-9367-f7c5a4bb5f30 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8c458863-3c1d-48fd-9367-f7c5a4bb5f30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:41.094495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:41.095460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:41.096824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:41.096824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:41.096824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:41.096824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:41.096824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:41.096824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:41.096824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:41.096824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:41.096824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 411/832] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:41.103724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c458863-3c1d-48fd-9367-f7c5a4bb5f30 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:41.110621 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c34ca-6e59-4d72-95b5-8978a570215f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:41.149027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3651723-89da-4e02-b1d3-4322abe5836f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:41.149860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651723-89da-4e02-b1d3-4322abe5836f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:41.150137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651723-89da-4e02-b1d3-4322abe5836f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:41.150408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651723-89da-4e02-b1d3-4322abe5836f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:41.156514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3651723-89da-4e02-b1d3-4322abe5836f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:41.156514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3651723-89da-4e02-b1d3-4322abe5836f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-c3651723-89da-4e02-b1d3-4322abe5836f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:41.169218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:41.170219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:41.171492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 422/833] 10.222.0.50 () {56 vars in 1132 bytes} [Fri Apr 29 18:30:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:41.176561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3651723-89da-4e02-b1d3-4322abe5836f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:41.272220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c458863-3c1d-48fd-9367-f7c5a4bb5f30 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:41.290304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3651723-89da-4e02-b1d3-4322abe5836f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:41.318950 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb1bf6-1ad2-4ef8-a010-eb70ba3abfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:41.321146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb1bf6-1ad2-4ef8-a010-eb70ba3abfcf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:41.321499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb1bf6-1ad2-4ef8-a010-eb70ba3abfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:41.323117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb1bf6-1ad2-4ef8-a010-eb70ba3abfcf None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:41.324011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb1bf6-1ad2-4ef8-a010-eb70ba3abfcf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:41.324324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb1bf6-1ad2-4ef8-a010-eb70ba3abfcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:41.328131 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c506d8b-f00d-4298-9371-57a6a58f74ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:41.328858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c506d8b-f00d-4298-9371-57a6a58f74ed None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:41.329139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c506d8b-f00d-4298-9371-57a6a58f74ed None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:41.329241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb1bf6-1ad2-4ef8-a010-eb70ba3abfcf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:41.329399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c506d8b-f00d-4298-9371-57a6a58f74ed None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:41.330124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c506d8b-f00d-4298-9371-57a6a58f74ed None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:41.330431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c506d8b-f00d-4298-9371-57a6a58f74ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:41.332087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c506d8b-f00d-4298-9371-57a6a58f74ed None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:41.343141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 423/834] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:30:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:41.344148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c506d8b-f00d-4298-9371-57a6a58f74ed None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:41.355905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 412/835] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:41.355905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb1bf6-1ad2-4ef8-a010-eb70ba3abfcf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:41.445348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c506d8b-f00d-4298-9371-57a6a58f74ed None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:41.477351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb1bf6-1ad2-4ef8-a010-eb70ba3abfcf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:41.482998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f278b3-e4fd-4b3e-b237-58bea08fbdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:41.483682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f278b3-e4fd-4b3e-b237-58bea08fbdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:41.483963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f278b3-e4fd-4b3e-b237-58bea08fbdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:41.484236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f278b3-e4fd-4b3e-b237-58bea08fbdb8 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:41.485005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f278b3-e4fd-4b3e-b237-58bea08fbdb8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:41.485308 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f278b3-e4fd-4b3e-b237-58bea08fbdb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:41.487695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f278b3-e4fd-4b3e-b237-58bea08fbdb8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:41.493802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 424/836] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:30:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:41.518404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f777f-ef04-4e9c-9bc0-a32cf8d80543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:41.519332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f777f-ef04-4e9c-9bc0-a32cf8d80543 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:41.519606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f777f-ef04-4e9c-9bc0-a32cf8d80543 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:41.519875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f777f-ef04-4e9c-9bc0-a32cf8d80543 None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/4cf46333cd034f5dbd0b78503de1f8e1/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:41.524928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f777f-ef04-4e9c-9bc0-a32cf8d80543 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=4cf46333cd034f5dbd0b78503de1f8e1, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:41.526466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f777f-ef04-4e9c-9bc0-a32cf8d80543 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:41.551171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 413/837] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:30:41 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/4cf46333cd034f5dbd0b78503de1f8e1/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:30:42.112547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2a68f-73fd-4ab1-8c96-ef92296a07e9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:42.112969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2a68f-73fd-4ab1-8c96-ef92296a07e9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:42.113155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2a68f-73fd-4ab1-8c96-ef92296a07e9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:42.114491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 425/838] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:42.120094 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508ea29-676c-4a97-a62e-63216a4d5faa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:42.120378 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508ea29-676c-4a97-a62e-63216a4d5faa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:42.120679 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508ea29-676c-4a97-a62e-63216a4d5faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:42.139432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5508ea29-676c-4a97-a62e-63216a4d5faa None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:42.140630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5508ea29-676c-4a97-a62e-63216a4d5faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:42.211594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 414/839] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:30:42.234692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2a68f-73fd-4ab1-8c96-ef92296a07e9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:42.361263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2a68f-73fd-4ab1-8c96-ef92296a07e9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:42.396180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197372d-e901-46e2-b30f-0a3419c1ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:42.397308 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197372d-e901-46e2-b30f-0a3419c1ef3f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:42.397736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197372d-e901-46e2-b30f-0a3419c1ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:42.398147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197372d-e901-46e2-b30f-0a3419c1ef3f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:42.398991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197372d-e901-46e2-b30f-0a3419c1ef3f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:42.400745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197372d-e901-46e2-b30f-0a3419c1ef3f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-4197372d-e901-46e2-b30f-0a3419c1ef3f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:42.412945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:42.413981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Apr 29 18:30:42.415007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:42.417904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 426/840] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:30:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:42.428566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508ea29-676c-4a97-a62e-63216a4d5faa None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:42.539461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508ea29-676c-4a97-a62e-63216a4d5faa None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:42.575578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064da7d3-a878-4e22-b918-156ec9e94c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:42.576258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064da7d3-a878-4e22-b918-156ec9e94c6f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:42.580536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064da7d3-a878-4e22-b918-156ec9e94c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:42.580536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064da7d3-a878-4e22-b918-156ec9e94c6f None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:42.580536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064da7d3-a878-4e22-b918-156ec9e94c6f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:42.580536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064da7d3-a878-4e22-b918-156ec9e94c6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:42.580536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064da7d3-a878-4e22-b918-156ec9e94c6f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:42.585621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 415/841] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:30:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:42.592802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197372d-e901-46e2-b30f-0a3419c1ef3f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:42.653029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6465a-8393-4239-b0ba-31a02eadd73d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:42.653205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6465a-8393-4239-b0ba-31a02eadd73d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:42.655629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6465a-8393-4239-b0ba-31a02eadd73d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:42.655629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 416/842] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:42.661096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7eaa-ba0c-4d66-8c16-430244f6518a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:42.661372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7eaa-ba0c-4d66-8c16-430244f6518a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:42.661646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7eaa-ba0c-4d66-8c16-430244f6518a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:42.680101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ffbf7eaa-ba0c-4d66-8c16-430244f6518a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:42.681316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ffbf7eaa-ba0c-4d66-8c16-430244f6518a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:42.708146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197372d-e901-46e2-b30f-0a3419c1ef3f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:42.742716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ee470-6b02-4252-a24a-bc9e3935ae1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:42.743392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ee470-6b02-4252-a24a-bc9e3935ae1b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:42.743671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ee470-6b02-4252-a24a-bc9e3935ae1b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:42.743943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ee470-6b02-4252-a24a-bc9e3935ae1b None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:42.744697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ee470-6b02-4252-a24a-bc9e3935ae1b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:42.745001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ee470-6b02-4252-a24a-bc9e3935ae1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:42.746383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ee470-6b02-4252-a24a-bc9e3935ae1b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:42.751744 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 427/843] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:30:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:42.755704 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 417/844] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:42.779679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ee470-6b02-4252-a24a-bc9e3935ae1b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:42.912157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ee470-6b02-4252-a24a-bc9e3935ae1b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:42.947988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77c325-8b47-4acd-a95a-1969f430130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:42.948836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77c325-8b47-4acd-a95a-1969f430130c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:42.949113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77c325-8b47-4acd-a95a-1969f430130c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:42.952519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77c325-8b47-4acd-a95a-1969f430130c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:42.960779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77c325-8b47-4acd-a95a-1969f430130c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-4c77c325-8b47-4acd-a95a-1969f430130c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:42.969578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:42.970579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:42.971426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 428/845] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:30:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:42.981423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7eaa-ba0c-4d66-8c16-430244f6518a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:43.097049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7eaa-ba0c-4d66-8c16-430244f6518a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:43.136550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d83c0-53f5-4218-be0a-38982f2138ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:43.136550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d83c0-53f5-4218-be0a-38982f2138ed None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:43.136550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d83c0-53f5-4218-be0a-38982f2138ed None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:43.136550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d83c0-53f5-4218-be0a-38982f2138ed None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:43.136966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d83c0-53f5-4218-be0a-38982f2138ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:43.136966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d83c0-53f5-4218-be0a-38982f2138ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:43.136966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d83c0-53f5-4218-be0a-38982f2138ed None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:43.149670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 418/846] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:30:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:43.158350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77c325-8b47-4acd-a95a-1969f430130c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:43.267030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77c325-8b47-4acd-a95a-1969f430130c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:43.301134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3a618-a66c-46d3-ba60-5267cd6fd06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:43.302340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3a618-a66c-46d3-ba60-5267cd6fd06e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:43.302815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3a618-a66c-46d3-ba60-5267cd6fd06e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:43.303279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3a618-a66c-46d3-ba60-5267cd6fd06e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:43.314627 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3a618-a66c-46d3-ba60-5267cd6fd06e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:43.316470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3a618-a66c-46d3-ba60-5267cd6fd06e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-1fc3a618-a66c-46d3-ba60-5267cd6fd06e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:43.337716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:43.338654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:43.339569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:43.339569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:43.339569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:43.339569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:43.342190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 429/847] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:43.348076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d83c0-53f5-4218-be0a-38982f2138ed None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:43.460019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d83c0-53f5-4218-be0a-38982f2138ed None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:43.500459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d06ce-6441-447e-9e43-d91edf628052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:43.501558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d06ce-6441-447e-9e43-d91edf628052 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:43.502053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d06ce-6441-447e-9e43-d91edf628052 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:43.502509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d06ce-6441-447e-9e43-d91edf628052 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:43.503592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d06ce-6441-447e-9e43-d91edf628052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:43.504088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d06ce-6441-447e-9e43-d91edf628052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:43.505680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d06ce-6441-447e-9e43-d91edf628052 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:43.518637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 419/848] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:43.524769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3a618-a66c-46d3-ba60-5267cd6fd06e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:43.683698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3a618-a66c-46d3-ba60-5267cd6fd06e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:43.721022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c3e3f-d8ac-46f6-9514-a2c95c7ab192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:43.722106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c3e3f-d8ac-46f6-9514-a2c95c7ab192 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:43.722533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c3e3f-d8ac-46f6-9514-a2c95c7ab192 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:43.722976 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c3e3f-d8ac-46f6-9514-a2c95c7ab192 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:43.731457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c3e3f-d8ac-46f6-9514-a2c95c7ab192 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:43.733141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c3e3f-d8ac-46f6-9514-a2c95c7ab192 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-b21c3e3f-d8ac-46f6-9514-a2c95c7ab192 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:43.750082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:43.751328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:43.752560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:43.752560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:43.752560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:43.752560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:43.752560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:43.752560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:43.752560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:43.752560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:43.752560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:30:43.752560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:43.755036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 430/849] 10.222.0.50 () {56 vars in 1117 bytes} [Fri Apr 29 18:30:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:43.761097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d06ce-6441-447e-9e43-d91edf628052 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:43.902278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d06ce-6441-447e-9e43-d91edf628052 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:43.942761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7a44e-762a-44fc-8a2d-f4d64630e63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:43.943600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7a44e-762a-44fc-8a2d-f4d64630e63a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:43.944004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7a44e-762a-44fc-8a2d-f4d64630e63a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:43.944353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7a44e-762a-44fc-8a2d-f4d64630e63a None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:43.945723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7a44e-762a-44fc-8a2d-f4d64630e63a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:43.946120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7a44e-762a-44fc-8a2d-f4d64630e63a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:43.947675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7a44e-762a-44fc-8a2d-f4d64630e63a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:43.967798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 420/850] 10.222.0.50 () {56 vars in 1149 bytes} [Fri Apr 29 18:30:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:43.977780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c3e3f-d8ac-46f6-9514-a2c95c7ab192 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:44.079421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a480992-d605-47a1-99d0-f33462258636 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.079863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a480992-d605-47a1-99d0-f33462258636 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.080230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a480992-d605-47a1-99d0-f33462258636 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.085568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 421/851] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:44.086927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e699ebe-b740-452a-8d64-538ec52af55a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.087207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e699ebe-b740-452a-8d64-538ec52af55a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.087474 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e699ebe-b740-452a-8d64-538ec52af55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.103225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c3e3f-d8ac-46f6-9514-a2c95c7ab192 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:44.108067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1e699ebe-b740-452a-8d64-538ec52af55a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:44.109288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1e699ebe-b740-452a-8d64-538ec52af55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:44.140666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2ce1a-28bf-46c3-bb2b-725bf17e4d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:44.141721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2ce1a-28bf-46c3-bb2b-725bf17e4d9e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.142364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2ce1a-28bf-46c3-bb2b-725bf17e4d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.142932 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2ce1a-28bf-46c3-bb2b-725bf17e4d9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.187505 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 422/852] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:30:44.188919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a317b4-d0d0-48b3-ade5-cc9336fc0c44 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.189269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a317b4-d0d0-48b3-ade5-cc9336fc0c44 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.189606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a317b4-d0d0-48b3-ade5-cc9336fc0c44 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.194985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 423/853] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:44.196592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44092ac-b38d-4c98-bf34-0ef6e35216fd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.196907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44092ac-b38d-4c98-bf34-0ef6e35216fd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.197187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44092ac-b38d-4c98-bf34-0ef6e35216fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.214252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f44092ac-b38d-4c98-bf34-0ef6e35216fd None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:44.215412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f44092ac-b38d-4c98-bf34-0ef6e35216fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:44.233726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2ce1a-28bf-46c3-bb2b-725bf17e4d9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:44.235209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2ce1a-28bf-46c3-bb2b-725bf17e4d9e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:44.316873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 424/854] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:44.318116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44092ac-b38d-4c98-bf34-0ef6e35216fd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:44.357946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 431/855] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:30:44.358795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2ce1a-28bf-46c3-bb2b-725bf17e4d9e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:44.456285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44092ac-b38d-4c98-bf34-0ef6e35216fd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:44.478382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2ce1a-28bf-46c3-bb2b-725bf17e4d9e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:44.503589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85954f7-8b40-45fd-b34d-df1da86828df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:44.505163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85954f7-8b40-45fd-b34d-df1da86828df None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.505761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85954f7-8b40-45fd-b34d-df1da86828df None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.506191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85954f7-8b40-45fd-b34d-df1da86828df None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.507383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85954f7-8b40-45fd-b34d-df1da86828df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:44.509250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85954f7-8b40-45fd-b34d-df1da86828df None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:44.518688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627d12c-a41c-4f3e-b31e-bf15a1d82eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:44.519603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627d12c-a41c-4f3e-b31e-bf15a1d82eed None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.519881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627d12c-a41c-4f3e-b31e-bf15a1d82eed None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.520888 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627d12c-a41c-4f3e-b31e-bf15a1d82eed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.526759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 425/856] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:30:44 2022] POST /identity/v3/services => generated 269 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:44.528235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85954f7-8b40-45fd-b34d-df1da86828df None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:44.537749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627d12c-a41c-4f3e-b31e-bf15a1d82eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2627d12c-a41c-4f3e-b31e-bf15a1d82eed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:44.549209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:44.550368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:44.552988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 432/857] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:44.563630 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627d12c-a41c-4f3e-b31e-bf15a1d82eed None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:44.660179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85954f7-8b40-45fd-b34d-df1da86828df None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:44.700849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce106e42-483d-41a1-9016-01fbae715f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:44.701651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce106e42-483d-41a1-9016-01fbae715f79 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.702013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce106e42-483d-41a1-9016-01fbae715f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.702184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce106e42-483d-41a1-9016-01fbae715f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.711294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627d12c-a41c-4f3e-b31e-bf15a1d82eed None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:44.715995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce106e42-483d-41a1-9016-01fbae715f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:44.717297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce106e42-483d-41a1-9016-01fbae715f79 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ce106e42-483d-41a1-9016-01fbae715f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:44.740243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:44.741306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:44.742209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:44.742209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:44.742209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:44.742209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:44.748674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 426/858] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:44.753029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce106e42-483d-41a1-9016-01fbae715f79 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:44.762087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87d487-3aa7-42aa-bc67-4be0843e8f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:44.763344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87d487-3aa7-42aa-bc67-4be0843e8f97 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.764036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87d487-3aa7-42aa-bc67-4be0843e8f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.764530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87d487-3aa7-42aa-bc67-4be0843e8f97 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.765487 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87d487-3aa7-42aa-bc67-4be0843e8f97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:44.765996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87d487-3aa7-42aa-bc67-4be0843e8f97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:44.767588 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87d487-3aa7-42aa-bc67-4be0843e8f97 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:44.779494 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 433/859] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:44.785195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87d487-3aa7-42aa-bc67-4be0843e8f97 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:44.890172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce106e42-483d-41a1-9016-01fbae715f79 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:44.931410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b709c-3c85-48cc-be53-e42ec0ae5bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:44.932428 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b709c-3c85-48cc-be53-e42ec0ae5bdb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.933089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b709c-3c85-48cc-be53-e42ec0ae5bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.933515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b709c-3c85-48cc-be53-e42ec0ae5bdb None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.935159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b709c-3c85-48cc-be53-e42ec0ae5bdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:44.935633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b709c-3c85-48cc-be53-e42ec0ae5bdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:44.937206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b709c-3c85-48cc-be53-e42ec0ae5bdb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:44.939392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87d487-3aa7-42aa-bc67-4be0843e8f97 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:44.952679 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 427/860] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:44.965640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b709c-3c85-48cc-be53-e42ec0ae5bdb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:44.984027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f9508-00d3-49d3-aff4-71d613ce2b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:44.984866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f9508-00d3-49d3-aff4-71d613ce2b3f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:44.985140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f9508-00d3-49d3-aff4-71d613ce2b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:44.985402 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f9508-00d3-49d3-aff4-71d613ce2b3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:44.997570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f9508-00d3-49d3-aff4-71d613ce2b3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:45.000573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f9508-00d3-49d3-aff4-71d613ce2b3f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-aa2f9508-00d3-49d3-aff4-71d613ce2b3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:45.026234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:45.027245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:45.028146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:45.028146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:45.028146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:45.028146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:45.035320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 434/861] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:45.037784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f9508-00d3-49d3-aff4-71d613ce2b3f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:45.197116 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f9508-00d3-49d3-aff4-71d613ce2b3f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:45.240066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b15e-c0b1-4ce6-9dda-87f074428262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:45.242128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b709c-3c85-48cc-be53-e42ec0ae5bdb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:45.243102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b15e-c0b1-4ce6-9dda-87f074428262 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:45.243546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b15e-c0b1-4ce6-9dda-87f074428262 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:45.243964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b15e-c0b1-4ce6-9dda-87f074428262 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:45.245949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b15e-c0b1-4ce6-9dda-87f074428262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:45.246409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b15e-c0b1-4ce6-9dda-87f074428262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:45.247945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b15e-c0b1-4ce6-9dda-87f074428262 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:45.262863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 435/862] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:45.269426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b15e-c0b1-4ce6-9dda-87f074428262 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:45.304356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576eb0a-05ab-45e0-b005-a7aa11ad043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:45.309520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576eb0a-05ab-45e0-b005-a7aa11ad043e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:45.310060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576eb0a-05ab-45e0-b005-a7aa11ad043e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:45.310537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576eb0a-05ab-45e0-b005-a7aa11ad043e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:45.325744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576eb0a-05ab-45e0-b005-a7aa11ad043e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:45.328513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576eb0a-05ab-45e0-b005-a7aa11ad043e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8576eb0a-05ab-45e0-b005-a7aa11ad043e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:45.350656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:45.351678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:45.352707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:45.352707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:45.352707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:45.352707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:45.352707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:45.352707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:45.352707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:45.352707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:45.352707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 428/863] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:45.354378 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d268366-7802-4db8-ac44-9706ddb27a77 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:45.354378 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d268366-7802-4db8-ac44-9706ddb27a77 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:45.354378 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d268366-7802-4db8-ac44-9706ddb27a77 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:45.359379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 429/864] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:45.360648 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f391e3-25c2-40d2-860d-ccdb8fb3ec85 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:45.361008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f391e3-25c2-40d2-860d-ccdb8fb3ec85 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:45.361292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f391e3-25c2-40d2-860d-ccdb8fb3ec85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:45.379128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-34f391e3-25c2-40d2-860d-ccdb8fb3ec85 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:45.380309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34f391e3-25c2-40d2-860d-ccdb8fb3ec85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:45.422437 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b15e-c0b1-4ce6-9dda-87f074428262 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:45.460395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 430/865] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:45 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:45.462007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f391e3-25c2-40d2-860d-ccdb8fb3ec85 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:45.464304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e2c34-da8f-4909-9721-d5d45b155b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:45.465229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e2c34-da8f-4909-9721-d5d45b155b2e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:45.465568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e2c34-da8f-4909-9721-d5d45b155b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:45.465926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e2c34-da8f-4909-9721-d5d45b155b2e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:45.476768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e2c34-da8f-4909-9721-d5d45b155b2e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:45.478525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e2c34-da8f-4909-9721-d5d45b155b2e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2b0e2c34-da8f-4909-9721-d5d45b155b2e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:45.494973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:45.495957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:45.497367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:45.497367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:45.497367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:45.497367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:45.497367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:45.497367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:45.497367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:45.497367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:45.497367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Apr 29 18:30:45.497367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:45.500892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 436/866] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:30:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:45.501869 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e2c34-da8f-4909-9721-d5d45b155b2e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:45.599830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f391e3-25c2-40d2-860d-ccdb8fb3ec85 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:45.623475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e2c34-da8f-4909-9721-d5d45b155b2e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:45.644091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6987a-3134-41fb-8f77-256d6c2d135f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:45.644821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6987a-3134-41fb-8f77-256d6c2d135f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:45.645153 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6987a-3134-41fb-8f77-256d6c2d135f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:45.645365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6987a-3134-41fb-8f77-256d6c2d135f None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:45.646458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6987a-3134-41fb-8f77-256d6c2d135f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:45.646764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6987a-3134-41fb-8f77-256d6c2d135f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:45.648219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6987a-3134-41fb-8f77-256d6c2d135f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:45.668987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 431/867] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:45.668987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6987a-3134-41fb-8f77-256d6c2d135f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:45.671153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e373904-b694-45d8-a63d-6bd4f4d64162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:45.671891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e373904-b694-45d8-a63d-6bd4f4d64162 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:45.672177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e373904-b694-45d8-a63d-6bd4f4d64162 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:45.672451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e373904-b694-45d8-a63d-6bd4f4d64162 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:45.673244 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e373904-b694-45d8-a63d-6bd4f4d64162 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:45.675115 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e373904-b694-45d8-a63d-6bd4f4d64162 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:45.691495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 437/868] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:30:45 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:45.692316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e373904-b694-45d8-a63d-6bd4f4d64162 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:45.873900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6987a-3134-41fb-8f77-256d6c2d135f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:45.929172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e373904-b694-45d8-a63d-6bd4f4d64162 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:45.937347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4e334-aceb-4a0f-8445-7a744066871c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:45.938155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4e334-aceb-4a0f-8445-7a744066871c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:45.938438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4e334-aceb-4a0f-8445-7a744066871c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:45.939274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4e334-aceb-4a0f-8445-7a744066871c None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:45.940173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4e334-aceb-4a0f-8445-7a744066871c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:45.940508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4e334-aceb-4a0f-8445-7a744066871c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:45.943286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4e334-aceb-4a0f-8445-7a744066871c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:45.969808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 432/869] 10.222.0.50 () {56 vars in 1135 bytes} [Fri Apr 29 18:30:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:45.977985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4e334-aceb-4a0f-8445-7a744066871c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:45.993571 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6a273-eb30-4f59-af13-a39c62ff822b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:45.994342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6a273-eb30-4f59-af13-a39c62ff822b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:45.994508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6a273-eb30-4f59-af13-a39c62ff822b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:45.994776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6a273-eb30-4f59-af13-a39c62ff822b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:45.995601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6a273-eb30-4f59-af13-a39c62ff822b None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=b7edbbc857d942d685c124c2c6aac09e) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:45.995904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6a273-eb30-4f59-af13-a39c62ff822b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:45.997370 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6a273-eb30-4f59-af13-a39c62ff822b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:46.012515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 438/870] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:45 2022] GET /identity/v3/role_assignments?user.id=b7edbbc857d942d685c124c2c6aac09e&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:46.105391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4e334-aceb-4a0f-8445-7a744066871c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:46.142836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5eff1-bbac-4ebb-bff2-2709fd24197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:46.143420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5eff1-bbac-4ebb-bff2-2709fd24197b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:46.143694 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5eff1-bbac-4ebb-bff2-2709fd24197b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:46.144001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5eff1-bbac-4ebb-bff2-2709fd24197b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:46.219438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5eff1-bbac-4ebb-bff2-2709fd24197b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:46.220172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5eff1-bbac-4ebb-bff2-2709fd24197b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:46.328347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 433/871] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:45 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Apr 29 18:30:46.334825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6a273-eb30-4f59-af13-a39c62ff822b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:46.461084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6a273-eb30-4f59-af13-a39c62ff822b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:46.498712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9c7ca-5e47-45b5-9c11-551357836b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:46.499543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c7ca-5e47-45b5-9c11-551357836b86 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:46.499885 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c7ca-5e47-45b5-9c11-551357836b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:46.500166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c7ca-5e47-45b5-9c11-551357836b86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:46.511676 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9c7ca-5e47-45b5-9c11-551357836b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:46.513170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9c7ca-5e47-45b5-9c11-551357836b86 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d4d9c7ca-5e47-45b5-9c11-551357836b86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:46.530779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:46.531713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:46.532635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:46.532635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:46.532635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:46.532635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:46.532635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 439/872] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:46.541664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5eff1-bbac-4ebb-bff2-2709fd24197b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:46.668688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5eff1-bbac-4ebb-bff2-2709fd24197b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:46.710386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d1a3e-998a-44b3-8bb3-ddbf66e5e0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:46.710992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d1a3e-998a-44b3-8bb3-ddbf66e5e0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:46.711374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d1a3e-998a-44b3-8bb3-ddbf66e5e0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:46.711586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d1a3e-998a-44b3-8bb3-ddbf66e5e0e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:46.712313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d1a3e-998a-44b3-8bb3-ddbf66e5e0e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:46.712663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d1a3e-998a-44b3-8bb3-ddbf66e5e0e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:46.714963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d1a3e-998a-44b3-8bb3-ddbf66e5e0e9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:46.727946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 434/873] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:46.736610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9c7ca-5e47-45b5-9c11-551357836b86 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:46.854684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9c7ca-5e47-45b5-9c11-551357836b86 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:46.892994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf00fd-2ed5-4c02-82e5-51e0038078fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:46.894009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf00fd-2ed5-4c02-82e5-51e0038078fd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:46.894293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf00fd-2ed5-4c02-82e5-51e0038078fd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:46.894567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf00fd-2ed5-4c02-82e5-51e0038078fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:46.901715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf00fd-2ed5-4c02-82e5-51e0038078fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:46.903322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf00fd-2ed5-4c02-82e5-51e0038078fd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-40cf00fd-2ed5-4c02-82e5-51e0038078fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:46.918345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:46.919466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:46.920590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:46.920590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:46.920590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:46.920590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:46.920590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:46.920590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:46.920590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:46.920590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:46.920590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 440/874] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:46.933594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d1a3e-998a-44b3-8bb3-ddbf66e5e0e9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:47.046182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde79898-1770-413d-b160-15c6c5082567 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:47.046704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde79898-1770-413d-b160-15c6c5082567 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:47.046704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde79898-1770-413d-b160-15c6c5082567 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:47.048350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 441/875] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:47.054597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61280e-06b7-4b6b-8782-c5955b95e0d9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:47.055057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61280e-06b7-4b6b-8782-c5955b95e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:47.055403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61280e-06b7-4b6b-8782-c5955b95e0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:47.069386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d1a3e-998a-44b3-8bb3-ddbf66e5e0e9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:47.105251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8a49a-63e9-4c79-b692-097af020f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:47.105925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8a49a-63e9-4c79-b692-097af020f385 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:47.106222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8a49a-63e9-4c79-b692-097af020f385 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:47.106482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8a49a-63e9-4c79-b692-097af020f385 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:47.107271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8a49a-63e9-4c79-b692-097af020f385 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:47.107579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8a49a-63e9-4c79-b692-097af020f385 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:47.109060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8a49a-63e9-4c79-b692-097af020f385 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:47.125577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ed61280e-06b7-4b6b-8782-c5955b95e0d9 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:47.126780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ed61280e-06b7-4b6b-8782-c5955b95e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:47.127270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 435/876] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:47.132402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8a49a-63e9-4c79-b692-097af020f385 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:47.237075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8a49a-63e9-4c79-b692-097af020f385 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:47.258485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 442/877] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:47.275990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909b061-2bb4-40b3-9040-e1546b9ea83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:47.276688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909b061-2bb4-40b3-9040-e1546b9ea83d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:47.279366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909b061-2bb4-40b3-9040-e1546b9ea83d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:47.279366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909b061-2bb4-40b3-9040-e1546b9ea83d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:47.279366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909b061-2bb4-40b3-9040-e1546b9ea83d None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=4cf46333cd034f5dbd0b78503de1f8e1) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:47.279366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909b061-2bb4-40b3-9040-e1546b9ea83d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:47.279782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909b061-2bb4-40b3-9040-e1546b9ea83d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:47.292641 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 436/878] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:47 2022] GET /identity/v3/role_assignments?user.id=4cf46333cd034f5dbd0b78503de1f8e1&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=e55ef11844a742f5889243f189bd59bc => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:47.301779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61280e-06b7-4b6b-8782-c5955b95e0d9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:47.426250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61280e-06b7-4b6b-8782-c5955b95e0d9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:47.465158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259c2e9-6f60-4310-930a-30c3cb8d625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:47.466006 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259c2e9-6f60-4310-930a-30c3cb8d625c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:47.466286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259c2e9-6f60-4310-930a-30c3cb8d625c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:47.466567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259c2e9-6f60-4310-930a-30c3cb8d625c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:47.467578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259c2e9-6f60-4310-930a-30c3cb8d625c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:47.469001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259c2e9-6f60-4310-930a-30c3cb8d625c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-4259c2e9-6f60-4310-930a-30c3cb8d625c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:47.482010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:47.483089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:47.483949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 443/879] 10.222.0.50 () {56 vars in 1132 bytes} [Fri Apr 29 18:30:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:47.490494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909b061-2bb4-40b3-9040-e1546b9ea83d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:47.632232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909b061-2bb4-40b3-9040-e1546b9ea83d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:47.678824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b496689-b03f-40f6-b5e1-eeb24601e5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:47.680573 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b496689-b03f-40f6-b5e1-eeb24601e5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:47.680862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b496689-b03f-40f6-b5e1-eeb24601e5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:47.682135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b496689-b03f-40f6-b5e1-eeb24601e5a7 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:47.685444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b496689-b03f-40f6-b5e1-eeb24601e5a7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:47.685444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b496689-b03f-40f6-b5e1-eeb24601e5a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:47.685444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b496689-b03f-40f6-b5e1-eeb24601e5a7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:47.695209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 437/880] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:30:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:47.705755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259c2e9-6f60-4310-930a-30c3cb8d625c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:47.847698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259c2e9-6f60-4310-930a-30c3cb8d625c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:47.882176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba90552-cc39-4ea7-b109-16f3e3715bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:47.882777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba90552-cc39-4ea7-b109-16f3e3715bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:47.883451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba90552-cc39-4ea7-b109-16f3e3715bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:47.883905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba90552-cc39-4ea7-b109-16f3e3715bd9 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:47.884668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba90552-cc39-4ea7-b109-16f3e3715bd9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:47.884972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba90552-cc39-4ea7-b109-16f3e3715bd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:47.886362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba90552-cc39-4ea7-b109-16f3e3715bd9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:47.891847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 444/881] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:30:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:47.897428 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b496689-b03f-40f6-b5e1-eeb24601e5a7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:48.004354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b496689-b03f-40f6-b5e1-eeb24601e5a7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:48.041871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67675d4-845b-4312-b736-15c445f08f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:48.042541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67675d4-845b-4312-b736-15c445f08f9c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:48.042822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67675d4-845b-4312-b736-15c445f08f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:48.043097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67675d4-845b-4312-b736-15c445f08f9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:48.043896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67675d4-845b-4312-b736-15c445f08f9c None admin] RBAC: Adding query filter params (interface=public, service_id=bea19770f6ad431eafcaf37387743c25, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:48.044206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67675d4-845b-4312-b736-15c445f08f9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:48.045622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67675d4-845b-4312-b736-15c445f08f9c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:48.053885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 438/882] 10.222.0.50 () {56 vars in 1251 bytes} [Fri Apr 29 18:30:47 2022] GET /identity/v3/endpoints?service_id=bea19770f6ad431eafcaf37387743c25&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:48.059406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba90552-cc39-4ea7-b109-16f3e3715bd9 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:48.211431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba90552-cc39-4ea7-b109-16f3e3715bd9 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:48.245916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0da0-9e99-40e6-9685-e08d885b27d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:48.246595 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0da0-9e99-40e6-9685-e08d885b27d6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:48.246876 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0da0-9e99-40e6-9685-e08d885b27d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:48.247157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0da0-9e99-40e6-9685-e08d885b27d6 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:48.247839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f0da0-9e99-40e6-9685-e08d885b27d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:48.249248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f0da0-9e99-40e6-9685-e08d885b27d6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:48.255616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 445/883] 10.222.0.50 () {56 vars in 1087 bytes} [Fri Apr 29 18:30:48 2022] GET /identity/v3/services => generated 1268 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:30:48.335173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355744b1-b754-4813-99a9-8b1f6d30f737 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:48.335514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355744b1-b754-4813-99a9-8b1f6d30f737 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:48.335735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355744b1-b754-4813-99a9-8b1f6d30f737 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:48.337687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 439/884] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:48.343661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e9535-bc5a-4628-876c-d01067eacbfe None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:48.343943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e9535-bc5a-4628-876c-d01067eacbfe None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:48.344215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e9535-bc5a-4628-876c-d01067eacbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:48.372548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-171e9535-bc5a-4628-876c-d01067eacbfe None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:48.372548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-171e9535-bc5a-4628-876c-d01067eacbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:48.463035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 446/885] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:48.488810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355744b1-b754-4813-99a9-8b1f6d30f737 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:48.628790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355744b1-b754-4813-99a9-8b1f6d30f737 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:48.666709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726a102-d0ef-4e78-be00-4296715368a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:48.667782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726a102-d0ef-4e78-be00-4296715368a2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:48.668432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726a102-d0ef-4e78-be00-4296715368a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:48.669661 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726a102-d0ef-4e78-be00-4296715368a2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:48.670470 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726a102-d0ef-4e78-be00-4296715368a2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:48.671959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726a102-d0ef-4e78-be00-4296715368a2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f726a102-d0ef-4e78-be00-4296715368a2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:48.686246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:48.687821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Apr 29 18:30:48.688717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:48.691519 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 440/886] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:30:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:48.701954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e9535-bc5a-4628-876c-d01067eacbfe None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:48.726223 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68098ad4-7e1d-47db-b7d7-e607b4d3edc1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:48.726407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68098ad4-7e1d-47db-b7d7-e607b4d3edc1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:48.726742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68098ad4-7e1d-47db-b7d7-e607b4d3edc1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:48.728480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 441/887] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:30:48.733613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c48c1e-2584-4250-8616-a99e41654620 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:48.733903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c48c1e-2584-4250-8616-a99e41654620 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:48.734182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c48c1e-2584-4250-8616-a99e41654620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:48.762959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c8c48c1e-2584-4250-8616-a99e41654620 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:48.764161 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c8c48c1e-2584-4250-8616-a99e41654620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:48.858409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e9535-bc5a-4628-876c-d01067eacbfe None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:48.859260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 442/888] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:48.885750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c48c1e-2584-4250-8616-a99e41654620 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:48.938152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493a337-52b0-48ab-9482-4a2603806f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:48.939641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493a337-52b0-48ab-9482-4a2603806f02 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:48.940203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493a337-52b0-48ab-9482-4a2603806f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:48.942460 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493a337-52b0-48ab-9482-4a2603806f02 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:48.943319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493a337-52b0-48ab-9482-4a2603806f02 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:48.944406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493a337-52b0-48ab-9482-4a2603806f02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:48.946084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493a337-52b0-48ab-9482-4a2603806f02 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:48.962675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 447/889] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:30:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:48.964363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493a337-52b0-48ab-9482-4a2603806f02 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:49.081776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c48c1e-2584-4250-8616-a99e41654620 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:49.145770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ff8e4-2fbe-4516-8028-c13d3981acb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:49.146981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ff8e4-2fbe-4516-8028-c13d3981acb1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:49.146981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ff8e4-2fbe-4516-8028-c13d3981acb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:49.148120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ff8e4-2fbe-4516-8028-c13d3981acb1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:49.163371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493a337-52b0-48ab-9482-4a2603806f02 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:49.165999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ff8e4-2fbe-4516-8028-c13d3981acb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b49ff8e4-2fbe-4516-8028-c13d3981acb1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:49.175687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:49.176691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:30:49.177539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:49.188640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 443/890] 10.222.0.50 () {56 vars in 1120 bytes} [Fri Apr 29 18:30:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:49.189625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ff8e4-2fbe-4516-8028-c13d3981acb1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:49.223209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15953c5e-dcfe-441d-ab67-e9e4e5301c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:49.224312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15953c5e-dcfe-441d-ab67-e9e4e5301c58 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:49.224797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15953c5e-dcfe-441d-ab67-e9e4e5301c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:49.225261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15953c5e-dcfe-441d-ab67-e9e4e5301c58 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:49.226163 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15953c5e-dcfe-441d-ab67-e9e4e5301c58 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:49.227239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15953c5e-dcfe-441d-ab67-e9e4e5301c58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:49.228860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15953c5e-dcfe-441d-ab67-e9e4e5301c58 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:49.240032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 448/891] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:30:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:49.241374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15953c5e-dcfe-441d-ab67-e9e4e5301c58 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:49.321520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ff8e4-2fbe-4516-8028-c13d3981acb1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:49.359597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15953c5e-dcfe-441d-ab67-e9e4e5301c58 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:49.369670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69436688-a5f8-4de6-a5de-b33f8d0d08d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:49.370314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69436688-a5f8-4de6-a5de-b33f8d0d08d4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:49.370620 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69436688-a5f8-4de6-a5de-b33f8d0d08d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:49.370940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69436688-a5f8-4de6-a5de-b33f8d0d08d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:49.375503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69436688-a5f8-4de6-a5de-b33f8d0d08d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:49.375503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69436688-a5f8-4de6-a5de-b33f8d0d08d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:49.378916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69436688-a5f8-4de6-a5de-b33f8d0d08d4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:49.401925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 444/892] 10.222.0.50 () {56 vars in 1115 bytes} [Fri Apr 29 18:30:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:49.402854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69436688-a5f8-4de6-a5de-b33f8d0d08d4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:49.405033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a0998-f656-41db-8e9c-1cfc6c3a5189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:49.405988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a0998-f656-41db-8e9c-1cfc6c3a5189 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:49.406467 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a0998-f656-41db-8e9c-1cfc6c3a5189 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:49.407001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a0998-f656-41db-8e9c-1cfc6c3a5189 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:49.408085 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a0998-f656-41db-8e9c-1cfc6c3a5189 None admin] RBAC: Adding query filter params (interface=public, service_id=ce0c8927f1cc4d9eb279b9573cb9f403, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:49.408613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a0998-f656-41db-8e9c-1cfc6c3a5189 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:49.410222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a0998-f656-41db-8e9c-1cfc6c3a5189 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:49.420741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 449/893] 10.222.0.50 () {56 vars in 1251 bytes} [Fri Apr 29 18:30:49 2022] GET /identity/v3/endpoints?service_id=ce0c8927f1cc4d9eb279b9573cb9f403&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:49.423640 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a0998-f656-41db-8e9c-1cfc6c3a5189 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:49.540630 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a0998-f656-41db-8e9c-1cfc6c3a5189 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:49.562647 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69436688-a5f8-4de6-a5de-b33f8d0d08d4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:49.586856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf85fd-8700-4203-9763-278e70630512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:49.587973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf85fd-8700-4203-9763-278e70630512 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:49.588441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf85fd-8700-4203-9763-278e70630512 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:49.588988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf85fd-8700-4203-9763-278e70630512 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:49.589870 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf85fd-8700-4203-9763-278e70630512 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:49.591482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf85fd-8700-4203-9763-278e70630512 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:49.604830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 450/894] 10.222.0.50 () {56 vars in 1087 bytes} [Fri Apr 29 18:30:49 2022] GET /identity/v3/services => generated 1268 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:30:49.630390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed040a92-9997-4cdd-afac-4c5975900294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:49.631684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed040a92-9997-4cdd-afac-4c5975900294 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:49.632179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed040a92-9997-4cdd-afac-4c5975900294 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:49.632732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed040a92-9997-4cdd-afac-4c5975900294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:49.651318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed040a92-9997-4cdd-afac-4c5975900294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:49.653369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed040a92-9997-4cdd-afac-4c5975900294 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ed040a92-9997-4cdd-afac-4c5975900294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:49.673311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:49.674295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:49.675225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:49.675225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:49.675225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:49.675225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:49.677664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 445/895] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:49.683682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf85fd-8700-4203-9763-278e70630512 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:49.792599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf85fd-8700-4203-9763-278e70630512 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:49.830825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182df0db-09d6-486d-92b1-23f6f2074483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:49.831525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182df0db-09d6-486d-92b1-23f6f2074483 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:49.831806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182df0db-09d6-486d-92b1-23f6f2074483 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:49.832077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182df0db-09d6-486d-92b1-23f6f2074483 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:49.832857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182df0db-09d6-486d-92b1-23f6f2074483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:49.833163 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182df0db-09d6-486d-92b1-23f6f2074483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:49.834570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182df0db-09d6-486d-92b1-23f6f2074483 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:49.848184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f067a-ac19-4bee-945f-deb3fdab61f8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:49.848739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f067a-ac19-4bee-945f-deb3fdab61f8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:49.848843 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f067a-ac19-4bee-945f-deb3fdab61f8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:49.850324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 446/896] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:30:49.855210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 451/897] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:49.857643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097406b-8a87-4ad1-bdca-60327edd559d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:49.857935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097406b-8a87-4ad1-bdca-60327edd559d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:49.858215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097406b-8a87-4ad1-bdca-60327edd559d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:49.860246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182df0db-09d6-486d-92b1-23f6f2074483 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:49.882125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1097406b-8a87-4ad1-bdca-60327edd559d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:49.883729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1097406b-8a87-4ad1-bdca-60327edd559d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:50.000575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182df0db-09d6-486d-92b1-23f6f2074483 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:50.003927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 447/898] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:50.040097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097406b-8a87-4ad1-bdca-60327edd559d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:50.060516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04948c01-fac1-4888-9621-6ea6fd096fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:50.060516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04948c01-fac1-4888-9621-6ea6fd096fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:50.060516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04948c01-fac1-4888-9621-6ea6fd096fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:50.060516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04948c01-fac1-4888-9621-6ea6fd096fd4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:50.076525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04948c01-fac1-4888-9621-6ea6fd096fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:50.080232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04948c01-fac1-4888-9621-6ea6fd096fd4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-04948c01-fac1-4888-9621-6ea6fd096fd4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:50.096001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:50.097054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:50.098481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 452/899] 10.222.0.50 () {56 vars in 1117 bytes} [Fri Apr 29 18:30:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:50.112519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04948c01-fac1-4888-9621-6ea6fd096fd4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:50.180591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097406b-8a87-4ad1-bdca-60327edd559d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:50.217443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff55441-1709-43c5-b871-c67133bb383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:50.218331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff55441-1709-43c5-b871-c67133bb383f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:50.224513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff55441-1709-43c5-b871-c67133bb383f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:50.224513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff55441-1709-43c5-b871-c67133bb383f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:50.224513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff55441-1709-43c5-b871-c67133bb383f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:50.224513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff55441-1709-43c5-b871-c67133bb383f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8ff55441-1709-43c5-b871-c67133bb383f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:50.234724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:50.235725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:30:50.236615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:50.237883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 448/900] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:30:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:50.244813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff55441-1709-43c5-b871-c67133bb383f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:50.264912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04948c01-fac1-4888-9621-6ea6fd096fd4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:50.315851 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3777203-27e6-42b8-9616-ce6ee34753b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:50.317159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3777203-27e6-42b8-9616-ce6ee34753b8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:50.317639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3777203-27e6-42b8-9616-ce6ee34753b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:50.318106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3777203-27e6-42b8-9616-ce6ee34753b8 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:50.319189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3777203-27e6-42b8-9616-ce6ee34753b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:50.319709 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3777203-27e6-42b8-9616-ce6ee34753b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:50.321891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3777203-27e6-42b8-9616-ce6ee34753b8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:50.344764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 453/901] 10.222.0.50 () {56 vars in 1149 bytes} [Fri Apr 29 18:30:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:50.355049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3777203-27e6-42b8-9616-ce6ee34753b8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:50.400070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff55441-1709-43c5-b871-c67133bb383f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:50.447930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99850-61ae-4881-8d70-ee284d6ae14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:50.448660 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99850-61ae-4881-8d70-ee284d6ae14e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:50.448944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99850-61ae-4881-8d70-ee284d6ae14e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:50.449386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99850-61ae-4881-8d70-ee284d6ae14e None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:50.450097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99850-61ae-4881-8d70-ee284d6ae14e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:50.450397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99850-61ae-4881-8d70-ee284d6ae14e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:50.452065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99850-61ae-4881-8d70-ee284d6ae14e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:50.463871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3777203-27e6-42b8-9616-ce6ee34753b8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:50.465543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 449/902] 10.222.0.50 () {56 vars in 1122 bytes} [Fri Apr 29 18:30:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:50.466537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99850-61ae-4881-8d70-ee284d6ae14e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:50.515152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ee930-c036-4cdf-91aa-c6d69673bd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:50.515846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ee930-c036-4cdf-91aa-c6d69673bd51 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:50.516125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ee930-c036-4cdf-91aa-c6d69673bd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:50.516388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ee930-c036-4cdf-91aa-c6d69673bd51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:50.600820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99850-61ae-4881-8d70-ee284d6ae14e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:50.624538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ee930-c036-4cdf-91aa-c6d69673bd51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:50.624538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ee930-c036-4cdf-91aa-c6d69673bd51 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:50.700594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4180d5-96aa-4b7f-8b9a-b99bbfe025c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:50.700594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4180d5-96aa-4b7f-8b9a-b99bbfe025c1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:50.700594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4180d5-96aa-4b7f-8b9a-b99bbfe025c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:50.700594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4180d5-96aa-4b7f-8b9a-b99bbfe025c1 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:50.701022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4180d5-96aa-4b7f-8b9a-b99bbfe025c1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:50.701022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4180d5-96aa-4b7f-8b9a-b99bbfe025c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:50.701382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4180d5-96aa-4b7f-8b9a-b99bbfe025c1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:50.713714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 450/903] 10.222.0.50 () {56 vars in 1122 bytes} [Fri Apr 29 18:30:50 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:50.715561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401785c2-d7b5-4239-ac6b-330038867daa None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:50.716064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401785c2-d7b5-4239-ac6b-330038867daa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:50.716581 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401785c2-d7b5-4239-ac6b-330038867daa None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:50.723106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 451/904] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:50.724976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6cce8-602a-4d8e-8337-032ea657372c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:50.725486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6cce8-602a-4d8e-8337-032ea657372c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:50.725962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6cce8-602a-4d8e-8337-032ea657372c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:50.762810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-62a6cce8-602a-4d8e-8337-032ea657372c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:50.766443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-62a6cce8-602a-4d8e-8337-032ea657372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:50.826676 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 454/905] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:30:50 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:30:50.832541 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ee930-c036-4cdf-91aa-c6d69673bd51 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:50.888517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 452/906] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:50.913182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6cce8-602a-4d8e-8337-032ea657372c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:51.015235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ee930-c036-4cdf-91aa-c6d69673bd51 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:51.055597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e705e28-2353-41c8-9fe9-ad0fff5a9892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:51.057040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e705e28-2353-41c8-9fe9-ad0fff5a9892 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:51.057519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e705e28-2353-41c8-9fe9-ad0fff5a9892 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:51.057983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e705e28-2353-41c8-9fe9-ad0fff5a9892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:51.059335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6cce8-602a-4d8e-8337-032ea657372c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:51.080731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e705e28-2353-41c8-9fe9-ad0fff5a9892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:51.085300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e705e28-2353-41c8-9fe9-ad0fff5a9892 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:51.099329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1bfdd-0327-4ccd-9116-18d4d10ab6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:51.101695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1bfdd-0327-4ccd-9116-18d4d10ab6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:51.101985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1bfdd-0327-4ccd-9116-18d4d10ab6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:51.102496 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1bfdd-0327-4ccd-9116-18d4d10ab6f7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:51.102754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1bfdd-0327-4ccd-9116-18d4d10ab6f7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:51.110855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1bfdd-0327-4ccd-9116-18d4d10ab6f7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6e705e28-2353-41c8-9fe9-ad0fff5a9892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:51.127341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:51.128273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:51.129205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:51.129205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:51.129205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:51.129205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:51.131579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 455/907] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b7a1bfdd-0327-4ccd-9116-18d4d10ab6f7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:51.133049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:51.136763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:51.137684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 453/908] 10.222.0.50 () {56 vars in 1132 bytes} [Fri Apr 29 18:30:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:51.141686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e705e28-2353-41c8-9fe9-ad0fff5a9892 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:51.143937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1bfdd-0327-4ccd-9116-18d4d10ab6f7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:51.284930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1bfdd-0327-4ccd-9116-18d4d10ab6f7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:51.308003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e705e28-2353-41c8-9fe9-ad0fff5a9892 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:51.333574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dedd9-7329-4ca9-8e9a-4f3613cd8498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:51.334266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dedd9-7329-4ca9-8e9a-4f3613cd8498 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:51.334550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dedd9-7329-4ca9-8e9a-4f3613cd8498 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:51.337011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dedd9-7329-4ca9-8e9a-4f3613cd8498 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:51.337011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571dedd9-7329-4ca9-8e9a-4f3613cd8498 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:51.337342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571dedd9-7329-4ca9-8e9a-4f3613cd8498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:51.338752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571dedd9-7329-4ca9-8e9a-4f3613cd8498 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:51.351212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 454/909] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:30:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:51.358215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dedd9-7329-4ca9-8e9a-4f3613cd8498 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:51.368628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70037c76-c8a7-4faf-8da0-74e809d9070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:51.370052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70037c76-c8a7-4faf-8da0-74e809d9070a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:51.371392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70037c76-c8a7-4faf-8da0-74e809d9070a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:51.371864 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70037c76-c8a7-4faf-8da0-74e809d9070a None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:51.373658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70037c76-c8a7-4faf-8da0-74e809d9070a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:51.375615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70037c76-c8a7-4faf-8da0-74e809d9070a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:51.377599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70037c76-c8a7-4faf-8da0-74e809d9070a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:51.404392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 456/910] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:51.416526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70037c76-c8a7-4faf-8da0-74e809d9070a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:51.537692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70037c76-c8a7-4faf-8da0-74e809d9070a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:51.556517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dedd9-7329-4ca9-8e9a-4f3613cd8498 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:51.574455 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b2865-30e3-4ab9-91b5-9dcaeefe9888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:51.575279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b2865-30e3-4ab9-91b5-9dcaeefe9888 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:51.575543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b2865-30e3-4ab9-91b5-9dcaeefe9888 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:51.576412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b2865-30e3-4ab9-91b5-9dcaeefe9888 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:51.585862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b2865-30e3-4ab9-91b5-9dcaeefe9888 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:51.589174 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b2865-30e3-4ab9-91b5-9dcaeefe9888 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6e4b2865-30e3-4ab9-91b5-9dcaeefe9888 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:51.609766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:51.610782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:51.611714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:51.611714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:51.611714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:51.611714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:51.611714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:51.611714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:51.611714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:51.611714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:51.616485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 457/911] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:51.623482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b2865-30e3-4ab9-91b5-9dcaeefe9888 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:51.641336 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482df857-b712-4e6f-bfd6-bf9828f55c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:51.641336 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482df857-b712-4e6f-bfd6-bf9828f55c5f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:51.641336 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482df857-b712-4e6f-bfd6-bf9828f55c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:51.641336 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482df857-b712-4e6f-bfd6-bf9828f55c5f None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:51.641741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482df857-b712-4e6f-bfd6-bf9828f55c5f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:51.641741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482df857-b712-4e6f-bfd6-bf9828f55c5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:51.644528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482df857-b712-4e6f-bfd6-bf9828f55c5f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:51.656130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 455/912] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:30:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:51.662253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482df857-b712-4e6f-bfd6-bf9828f55c5f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:51.769294 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482df857-b712-4e6f-bfd6-bf9828f55c5f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:51.769515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b2865-30e3-4ab9-91b5-9dcaeefe9888 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:51.822796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef216c4-b7d4-43b7-a9b2-23431152dc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:51.823417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef216c4-b7d4-43b7-a9b2-23431152dc93 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:51.823739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef216c4-b7d4-43b7-a9b2-23431152dc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:51.824020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef216c4-b7d4-43b7-a9b2-23431152dc93 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:51.824874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef216c4-b7d4-43b7-a9b2-23431152dc93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:51.825179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef216c4-b7d4-43b7-a9b2-23431152dc93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:51.826677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef216c4-b7d4-43b7-a9b2-23431152dc93 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:51.833343 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddca55f-f053-404b-977a-58ab33e888fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:51.834046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddca55f-f053-404b-977a-58ab33e888fd None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:51.834319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddca55f-f053-404b-977a-58ab33e888fd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:51.834591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddca55f-f053-404b-977a-58ab33e888fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:51.835390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddca55f-f053-404b-977a-58ab33e888fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:51.838626 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddca55f-f053-404b-977a-58ab33e888fd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:51.858644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 458/913] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:51.861167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef216c4-b7d4-43b7-a9b2-23431152dc93 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:51.878829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 456/914] 10.222.0.50 () {60 vars in 1144 bytes} [Fri Apr 29 18:30:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:52.030830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328dc1a-18c3-450f-a0f6-6ab5d3bc2ed4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:52.031448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328dc1a-18c3-450f-a0f6-6ab5d3bc2ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:52.031448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328dc1a-18c3-450f-a0f6-6ab5d3bc2ed4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:52.032947 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 457/915] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:52.041010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfa547-1b89-4094-b1f7-df1d094fcac2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:52.041331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfa547-1b89-4094-b1f7-df1d094fcac2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:52.041563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfa547-1b89-4094-b1f7-df1d094fcac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:52.064986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef216c4-b7d4-43b7-a9b2-23431152dc93 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:52.077724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d6bfa547-1b89-4094-b1f7-df1d094fcac2 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:52.078851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6bfa547-1b89-4094-b1f7-df1d094fcac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:52.133184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2237fea-a666-444c-a5e5-4f0604db6f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:52.133925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2237fea-a666-444c-a5e5-4f0604db6f2c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:52.134204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2237fea-a666-444c-a5e5-4f0604db6f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:52.134471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2237fea-a666-444c-a5e5-4f0604db6f2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:52.135296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2237fea-a666-444c-a5e5-4f0604db6f2c None admin] RBAC: Adding query filter params (role.id=6f04dd8fb0cf4f47a384d0fcca2217ea, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=b7edbbc857d942d685c124c2c6aac09e) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:52.135597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2237fea-a666-444c-a5e5-4f0604db6f2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:52.137072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2237fea-a666-444c-a5e5-4f0604db6f2c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:52.152598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 459/916] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:30:51 2022] GET /identity/v3/role_assignments?user.id=b7edbbc857d942d685c124c2c6aac09e&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=6f04dd8fb0cf4f47a384d0fcca2217ea => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:52.187954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 458/917] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:52.215550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2237fea-a666-444c-a5e5-4f0604db6f2c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:52.348117 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2237fea-a666-444c-a5e5-4f0604db6f2c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:52.403744 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf82458-fa27-4054-a4f0-7a14b4c2fa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:52.404673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82458-fa27-4054-a4f0-7a14b4c2fa64 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:52.407556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82458-fa27-4054-a4f0-7a14b4c2fa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:52.407556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82458-fa27-4054-a4f0-7a14b4c2fa64 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:52.407556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf82458-fa27-4054-a4f0-7a14b4c2fa64 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:52.407556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf82458-fa27-4054-a4f0-7a14b4c2fa64 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-cdf82458-fa27-4054-a4f0-7a14b4c2fa64 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:52.424525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:52.425553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:52.426391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 460/918] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:30:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:52.432043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfa547-1b89-4094-b1f7-df1d094fcac2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:52.820436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfa547-1b89-4094-b1f7-df1d094fcac2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:52.854128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c48cee-e344-404e-8584-57c7b6771a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:52.854800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c48cee-e344-404e-8584-57c7b6771a5e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:52.855076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c48cee-e344-404e-8584-57c7b6771a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:52.855347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c48cee-e344-404e-8584-57c7b6771a5e None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:52.856051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c48cee-e344-404e-8584-57c7b6771a5e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:52.856354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c48cee-e344-404e-8584-57c7b6771a5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:52.857749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c48cee-e344-404e-8584-57c7b6771a5e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:52.862693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 459/919] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:30:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:52.868093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf82458-fa27-4054-a4f0-7a14b4c2fa64 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:52.963884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf82458-fa27-4054-a4f0-7a14b4c2fa64 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:52.999546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a8dc1-d928-4e77-bd90-22ff9fafac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:53.000246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a8dc1-d928-4e77-bd90-22ff9fafac91 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:53.000568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a8dc1-d928-4e77-bd90-22ff9fafac91 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:53.000846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a8dc1-d928-4e77-bd90-22ff9fafac91 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:53.001569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a8dc1-d928-4e77-bd90-22ff9fafac91 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:53.001871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a8dc1-d928-4e77-bd90-22ff9fafac91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:53.003304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a8dc1-d928-4e77-bd90-22ff9fafac91 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:53.009615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 461/920] 10.222.0.50 () {56 vars in 1114 bytes} [Fri Apr 29 18:30:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:53.015111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c48cee-e344-404e-8584-57c7b6771a5e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:53.116297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c48cee-e344-404e-8584-57c7b6771a5e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:53.149321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778e1e8-0ad1-4637-a9f7-203877dfabdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:53.149978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778e1e8-0ad1-4637-a9f7-203877dfabdb None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:53.150256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778e1e8-0ad1-4637-a9f7-203877dfabdb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:53.150523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778e1e8-0ad1-4637-a9f7-203877dfabdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:53.151274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778e1e8-0ad1-4637-a9f7-203877dfabdb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:53.152682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778e1e8-0ad1-4637-a9f7-203877dfabdb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:53.174198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 460/921] 10.222.0.50 () {60 vars in 1144 bytes} [Fri Apr 29 18:30:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:53.189798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66401237-3ff4-46e8-8336-dfb93313a7e0 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:53.189798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66401237-3ff4-46e8-8336-dfb93313a7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:53.189798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66401237-3ff4-46e8-8336-dfb93313a7e0 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:53.190720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 462/922] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:30:53.197500 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4c0b8-c3b5-44c5-a993-0a5c5ffedb3a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:53.197818 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4c0b8-c3b5-44c5-a993-0a5c5ffedb3a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:53.198049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4c0b8-c3b5-44c5-a993-0a5c5ffedb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:53.217073 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-09d4c0b8-c3b5-44c5-a993-0a5c5ffedb3a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:53.218241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-09d4c0b8-c3b5-44c5-a993-0a5c5ffedb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:53.298230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 461/923] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:53.321629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66401237-3ff4-46e8-8336-dfb93313a7e0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:53.431560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66401237-3ff4-46e8-8336-dfb93313a7e0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:53.467698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de47fa-c5ac-48a9-b77b-30ead70abd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:53.468840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de47fa-c5ac-48a9-b77b-30ead70abd11 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:53.469313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de47fa-c5ac-48a9-b77b-30ead70abd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:53.469771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de47fa-c5ac-48a9-b77b-30ead70abd11 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:53.470790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de47fa-c5ac-48a9-b77b-30ead70abd11 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:53.472375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de47fa-c5ac-48a9-b77b-30ead70abd11 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:53.490776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 463/924] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:30:53 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:54.571324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd4a99-b1c0-49ee-885c-7ab9bd2960b1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:54.571852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd4a99-b1c0-49ee-885c-7ab9bd2960b1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:54.571852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd4a99-b1c0-49ee-885c-7ab9bd2960b1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:54.573255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 462/925] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:54.579108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bc6e9-d630-43ac-81fa-aba3f88607b7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:54.579896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bc6e9-d630-43ac-81fa-aba3f88607b7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:54.580314 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bc6e9-d630-43ac-81fa-aba3f88607b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:54.600287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c47bc6e9-d630-43ac-81fa-aba3f88607b7 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:54.601915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c47bc6e9-d630-43ac-81fa-aba3f88607b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:54.689864 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 464/926] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:30:54.715703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd4a99-b1c0-49ee-885c-7ab9bd2960b1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:54.847318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd4a99-b1c0-49ee-885c-7ab9bd2960b1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:54.891884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e173c56-6efa-45dc-af0c-24f641298595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:54.892682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d8498-6fd2-48c5-a900-79a94e66245c None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:54.892817 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e173c56-6efa-45dc-af0c-24f641298595 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:54.892999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e173c56-6efa-45dc-af0c-24f641298595 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:54.893253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e173c56-6efa-45dc-af0c-24f641298595 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:54.893765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d8498-6fd2-48c5-a900-79a94e66245c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:54.894271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d8498-6fd2-48c5-a900-79a94e66245c None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:54.896081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 465/927] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:54.907424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e173c56-6efa-45dc-af0c-24f641298595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:54.908128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f29ca7-6539-4f54-8675-14fa926c4063 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:54.908658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f29ca7-6539-4f54-8675-14fa926c4063 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:54.909129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f29ca7-6539-4f54-8675-14fa926c4063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8e173c56-6efa-45dc-af0c-24f641298595 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:54.916542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:54.921151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:54.921995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 463/928] 10.222.0.50 () {56 vars in 1120 bytes} [Fri Apr 29 18:30:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:54.925628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e173c56-6efa-45dc-af0c-24f641298595 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:54.930954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c2f29ca7-6539-4f54-8675-14fa926c4063 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:54.932134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c2f29ca7-6539-4f54-8675-14fa926c4063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:55.012543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 466/929] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:30:55.036359 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e173c56-6efa-45dc-af0c-24f641298595 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:55.039157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f29ca7-6539-4f54-8675-14fa926c4063 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:55.073353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61624e37-a0d4-4af2-b4a2-dc9706ec840d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:55.074551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61624e37-a0d4-4af2-b4a2-dc9706ec840d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.075026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61624e37-a0d4-4af2-b4a2-dc9706ec840d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.075619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61624e37-a0d4-4af2-b4a2-dc9706ec840d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.076600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61624e37-a0d4-4af2-b4a2-dc9706ec840d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:55.077098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61624e37-a0d4-4af2-b4a2-dc9706ec840d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:55.078672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61624e37-a0d4-4af2-b4a2-dc9706ec840d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:55.092820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 464/930] 10.222.0.50 () {56 vars in 1115 bytes} [Fri Apr 29 18:30:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:55.094965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61624e37-a0d4-4af2-b4a2-dc9706ec840d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:55.138752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f29ca7-6539-4f54-8675-14fa926c4063 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:55.172811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9416f-d71b-4a06-a78f-431e6478d929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:55.173829 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9416f-d71b-4a06-a78f-431e6478d929 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.173941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9416f-d71b-4a06-a78f-431e6478d929 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.174148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9416f-d71b-4a06-a78f-431e6478d929 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.174803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9416f-d71b-4a06-a78f-431e6478d929 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:55.176184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9416f-d71b-4a06-a78f-431e6478d929 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-c2b9416f-d71b-4a06-a78f-431e6478d929 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:55.188311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:55.189568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:55.190618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 467/931] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:55.197514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9416f-d71b-4a06-a78f-431e6478d929 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:55.207674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61624e37-a0d4-4af2-b4a2-dc9706ec840d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:55.248225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448e07d-f9b7-4304-b7f5-a42b5fa25c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:55.249066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448e07d-f9b7-4304-b7f5-a42b5fa25c48 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.249347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448e07d-f9b7-4304-b7f5-a42b5fa25c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.249619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448e07d-f9b7-4304-b7f5-a42b5fa25c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.263613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448e07d-f9b7-4304-b7f5-a42b5fa25c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:55.265583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448e07d-f9b7-4304-b7f5-a42b5fa25c48 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a448e07d-f9b7-4304-b7f5-a42b5fa25c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:55.286354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:55.287350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:55.288249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:55.288249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:55.288249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:55.288249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:55.295941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 465/932] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:55.297137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448e07d-f9b7-4304-b7f5-a42b5fa25c48 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:55.315303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9416f-d71b-4a06-a78f-431e6478d929 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:55.349086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9d16a-fc89-4a1f-b848-2468772fc605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:55.349759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9d16a-fc89-4a1f-b848-2468772fc605 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.350038 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9d16a-fc89-4a1f-b848-2468772fc605 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.350307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9d16a-fc89-4a1f-b848-2468772fc605 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.351011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9d16a-fc89-4a1f-b848-2468772fc605 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:55.351311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9d16a-fc89-4a1f-b848-2468772fc605 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:55.353398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9d16a-fc89-4a1f-b848-2468772fc605 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:55.362918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 468/933] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:30:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:55.363950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9d16a-fc89-4a1f-b848-2468772fc605 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:55.414470 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448e07d-f9b7-4304-b7f5-a42b5fa25c48 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:55.456769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecfff4-946c-4428-a396-8448ef0b5ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:55.457499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecfff4-946c-4428-a396-8448ef0b5ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.457781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecfff4-946c-4428-a396-8448ef0b5ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.458048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecfff4-946c-4428-a396-8448ef0b5ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.458770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ecfff4-946c-4428-a396-8448ef0b5ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:55.459072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ecfff4-946c-4428-a396-8448ef0b5ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:55.461423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ecfff4-946c-4428-a396-8448ef0b5ca7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:55.482047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 466/934] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:55.482256 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9d16a-fc89-4a1f-b848-2468772fc605 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:55.483849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecfff4-946c-4428-a396-8448ef0b5ca7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:55.516575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23315-b018-4800-9960-d56e61309944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:55.517822 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23315-b018-4800-9960-d56e61309944 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.518405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23315-b018-4800-9960-d56e61309944 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.518893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23315-b018-4800-9960-d56e61309944 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.519817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d23315-b018-4800-9960-d56e61309944 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:55.520423 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d23315-b018-4800-9960-d56e61309944 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:55.522138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d23315-b018-4800-9960-d56e61309944 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:55.536939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 469/935] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:30:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:55.605082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecfff4-946c-4428-a396-8448ef0b5ca7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:55.659390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7680510-3541-44fc-80de-e7ecd46a6d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:55.660239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7680510-3541-44fc-80de-e7ecd46a6d4f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.660549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7680510-3541-44fc-80de-e7ecd46a6d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.660825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7680510-3541-44fc-80de-e7ecd46a6d4f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.670139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7680510-3541-44fc-80de-e7ecd46a6d4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:55.671539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7680510-3541-44fc-80de-e7ecd46a6d4f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e7680510-3541-44fc-80de-e7ecd46a6d4f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:55.688710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:55.689746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:55.690693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 467/936] 10.222.0.50 () {56 vars in 1117 bytes} [Fri Apr 29 18:30:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:55.697779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23315-b018-4800-9960-d56e61309944 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:55.722252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354e6b7-1e91-4d81-890d-229a693ad87f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.722621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354e6b7-1e91-4d81-890d-229a693ad87f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.722801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354e6b7-1e91-4d81-890d-229a693ad87f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.724464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 468/937] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:30:55.730935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9679a76-d296-4ef9-a09b-c0a188573442 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.731247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9679a76-d296-4ef9-a09b-c0a188573442 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.731487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9679a76-d296-4ef9-a09b-c0a188573442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.749465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e9679a76-d296-4ef9-a09b-c0a188573442 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:55.751324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e9679a76-d296-4ef9-a09b-c0a188573442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:55.804972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23315-b018-4800-9960-d56e61309944 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:55.836856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 469/938] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:30:55.837772 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60e9dd-1452-4bbe-be7c-33a4d374a825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:55.838462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60e9dd-1452-4bbe-be7c-33a4d374a825 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.838743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60e9dd-1452-4bbe-be7c-33a4d374a825 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.839017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60e9dd-1452-4bbe-be7c-33a4d374a825 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.839947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60e9dd-1452-4bbe-be7c-33a4d374a825 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:55.840250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60e9dd-1452-4bbe-be7c-33a4d374a825 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:55.842068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60e9dd-1452-4bbe-be7c-33a4d374a825 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:55.853417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9679a76-d296-4ef9-a09b-c0a188573442 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:55.859939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 470/939] 10.222.0.50 () {56 vars in 1149 bytes} [Fri Apr 29 18:30:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:55.865332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60e9dd-1452-4bbe-be7c-33a4d374a825 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:55.957392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9679a76-d296-4ef9-a09b-c0a188573442 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:55.964086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60e9dd-1452-4bbe-be7c-33a4d374a825 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:55.996859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:55.997528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.997807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.997972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4b9e7-cde3-4958-908e-89138ad5bb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:55.998083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:55.998718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4b9e7-cde3-4958-908e-89138ad5bb39 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:55.998816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:55.999048 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4b9e7-cde3-4958-908e-89138ad5bb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:55.999318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4b9e7-cde3-4958-908e-89138ad5bb39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.000179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.002060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WARNING keystone.server.flask.common [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Apr 29 18:30:56.009234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4b9e7-cde3-4958-908e-89138ad5bb39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:56.010655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4b9e7-cde3-4958-908e-89138ad5bb39 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.022888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.sql.core [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Apr 29 18:30:56.022888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Apr 29 18:30:56.022888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [parameters: {'id': '2dc555b30e9d4ccfa409a0d7cd62e576', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Apr 29 18:30:56.022888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-82d4b9e7-cde3-4958-908e-89138ad5bb39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:56.027790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:56.028877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:56.029849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:56.029849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:56.029849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:56.029849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2dc555b30e9d4ccfa409a0d7cd62e576.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2dc555b30e9d4ccfa409a0d7cd62e576. Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Apr 29 18:30:56.031461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Apr 29 18:30:56.032423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return project_ref.to_dict() Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application yield resource Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.rollback() Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.force_reraise() Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise self.value Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application session.commit() Apr 29 18:30:56.034151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1431, in commit Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._prepare_impl() Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.flush() Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3363, in flush Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._flush(objects) Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3503, in _flush Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application compat.raise_( Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3463, in _flush Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application flush_context.execute() Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Apr 29 18:30:56.035072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rec.execute(self) Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application _emit_insert_statements( Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application c = connection._execute_20( Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self._execute_context( Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Apr 29 18:30:56.035990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Apr 29 18:30:56.040908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [parameters: {'id': '2dc555b30e9d4ccfa409a0d7cd62e576', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:56.041827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:56.042629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:56.043521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:56.043521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Apr 29 18:30:56.043521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Apr 29 18:30:56.043521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:56.043521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:56.043521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Apr 29 18:30:56.043521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict( Apr 29 18:30:56.043521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2dc555b30e9d4ccfa409a0d7cd62e576. Apr 29 18:30:56.043521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.044087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 471/940] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:56.044087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e07cb-a2da-4211-8ba8-fa80718acd79 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.044087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e07cb-a2da-4211-8ba8-fa80718acd79 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.044087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e07cb-a2da-4211-8ba8-fa80718acd79 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.044087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 472/941] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:56.044087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e07cb-a2da-4211-8ba8-fa80718acd79 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:56.044442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 470/942] 10.222.0.50 () {60 vars in 1140 bytes} [Fri Apr 29 18:30:55 2022] POST /identity/v3/projects => generated 262 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:56.044442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:56.141365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa5da4-5984-41e1-81ff-b3ce4cdebdae None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:56.144660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e07cb-a2da-4211-8ba8-fa80718acd79 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:56.174594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a51cd-73b1-4345-ad33-e2aa9cc212e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:56.175391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a51cd-73b1-4345-ad33-e2aa9cc212e6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.175711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a51cd-73b1-4345-ad33-e2aa9cc212e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.175936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a51cd-73b1-4345-ad33-e2aa9cc212e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.176960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cf3ad-9893-406e-9621-fe75ab3041b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:56.177623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cf3ad-9893-406e-9621-fe75ab3041b2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.177902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cf3ad-9893-406e-9621-fe75ab3041b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.178175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cf3ad-9893-406e-9621-fe75ab3041b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.178934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cf3ad-9893-406e-9621-fe75ab3041b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:56.179241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cf3ad-9893-406e-9621-fe75ab3041b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:56.182195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cf3ad-9893-406e-9621-fe75ab3041b2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.183261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a51cd-73b1-4345-ad33-e2aa9cc212e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:56.185650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a51cd-73b1-4345-ad33-e2aa9cc212e6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.193173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 473/943] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:56.194454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442352-de3f-4123-b0c5-72c5b6aeb9e0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.194743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442352-de3f-4123-b0c5-72c5b6aeb9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.195070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442352-de3f-4123-b0c5-72c5b6aeb9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a54a51cd-73b1-4345-ad33-e2aa9cc212e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:56.203791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:56.204859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:56.205777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:56.205777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:56.205777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:56.205777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:56.205777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:56.205777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:56.205777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:56.205777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.206200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 471/944] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:56.206905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a51cd-73b1-4345-ad33-e2aa9cc212e6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:56.217558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-87442352-de3f-4123-b0c5-72c5b6aeb9e0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:56.218675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-87442352-de3f-4123-b0c5-72c5b6aeb9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:56.296517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 474/945] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:56.296517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442352-de3f-4123-b0c5-72c5b6aeb9e0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:56.320853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a51cd-73b1-4345-ad33-e2aa9cc212e6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:56.354559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59f093-bd9f-4067-b681-7f51629f59b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:56.355402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59f093-bd9f-4067-b681-7f51629f59b2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.355692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59f093-bd9f-4067-b681-7f51629f59b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.355967 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59f093-bd9f-4067-b681-7f51629f59b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.362642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59f093-bd9f-4067-b681-7f51629f59b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:56.364087 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59f093-bd9f-4067-b681-7f51629f59b2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ca59f093-bd9f-4067-b681-7f51629f59b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:30:56.378187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:56.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:56.380210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:30:56.380210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:30:56.380210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:30:56.380210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:30:56.380210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:30:56.380210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:30:56.380210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:30:56.380210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.380210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 472/946] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:30:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:56.380210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59f093-bd9f-4067-b681-7f51629f59b2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:56.387083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442352-de3f-4123-b0c5-72c5b6aeb9e0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:56.420868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b59062-1e21-4d1d-897e-93961a895369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:56.421519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b59062-1e21-4d1d-897e-93961a895369 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.421800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b59062-1e21-4d1d-897e-93961a895369 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.422073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b59062-1e21-4d1d-897e-93961a895369 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.422829 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b59062-1e21-4d1d-897e-93961a895369 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:56.423135 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b59062-1e21-4d1d-897e-93961a895369 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:56.424597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b59062-1e21-4d1d-897e-93961a895369 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.452535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 475/947] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:30:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:56.452535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b59062-1e21-4d1d-897e-93961a895369 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:56.495132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59f093-bd9f-4067-b681-7f51629f59b2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:56.540120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b38a4-94dc-419d-b0e0-3cd447f975a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:56.541373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b38a4-94dc-419d-b0e0-3cd447f975a5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.541751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b38a4-94dc-419d-b0e0-3cd447f975a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.542097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b38a4-94dc-419d-b0e0-3cd447f975a5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.543020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b38a4-94dc-419d-b0e0-3cd447f975a5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:56.544814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b38a4-94dc-419d-b0e0-3cd447f975a5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e63b38a4-94dc-419d-b0e0-3cd447f975a5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:56.561537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:56.562621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:30:56.563722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:56.564655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b59062-1e21-4d1d-897e-93961a895369 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:56.564743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 473/948] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:30:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:56.571562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b38a4-94dc-419d-b0e0-3cd447f975a5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:56.603697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e4042-87c0-4ac0-a299-92d81f473dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:56.604480 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e4042-87c0-4ac0-a299-92d81f473dee None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.604810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e4042-87c0-4ac0-a299-92d81f473dee None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.605152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e4042-87c0-4ac0-a299-92d81f473dee None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.606010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e4042-87c0-4ac0-a299-92d81f473dee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:56.606264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e4042-87c0-4ac0-a299-92d81f473dee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:56.607897 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e4042-87c0-4ac0-a299-92d81f473dee None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.626666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 476/949] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:30:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:56.629602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e4042-87c0-4ac0-a299-92d81f473dee None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:56.702454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b38a4-94dc-419d-b0e0-3cd447f975a5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:56.742509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf86aac-5bb3-4c01-a180-53bfa1efaece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:56.743583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf86aac-5bb3-4c01-a180-53bfa1efaece None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.744072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf86aac-5bb3-4c01-a180-53bfa1efaece None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.744564 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf86aac-5bb3-4c01-a180-53bfa1efaece None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.744991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e4042-87c0-4ac0-a299-92d81f473dee None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:56.746059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf86aac-5bb3-4c01-a180-53bfa1efaece None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:56.746573 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf86aac-5bb3-4c01-a180-53bfa1efaece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:56.748143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf86aac-5bb3-4c01-a180-53bfa1efaece None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.760751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 474/950] 10.222.0.50 () {56 vars in 1122 bytes} [Fri Apr 29 18:30:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:56.762929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf86aac-5bb3-4c01-a180-53bfa1efaece None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:56.778913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea700002-1353-4fe4-aa0d-da06277ce508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:56.779858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea700002-1353-4fe4-aa0d-da06277ce508 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.780177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea700002-1353-4fe4-aa0d-da06277ce508 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.780516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea700002-1353-4fe4-aa0d-da06277ce508 None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/b7edbbc857d942d685c124c2c6aac09e/roles/6f04dd8fb0cf4f47a384d0fcca2217ea` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.782977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea700002-1353-4fe4-aa0d-da06277ce508 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=b7edbbc857d942d685c124c2c6aac09e, role_id=6f04dd8fb0cf4f47a384d0fcca2217ea)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:56.784455 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea700002-1353-4fe4-aa0d-da06277ce508 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.804541 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 477/951] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:30:56 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/b7edbbc857d942d685c124c2c6aac09e/roles/6f04dd8fb0cf4f47a384d0fcca2217ea => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:30:56.879554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf86aac-5bb3-4c01-a180-53bfa1efaece None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:56.918539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4325b-8125-4ab2-9b9c-b7b71fedbb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:56.919271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4325b-8125-4ab2-9b9c-b7b71fedbb5d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:56.919552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4325b-8125-4ab2-9b9c-b7b71fedbb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:56.919877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4325b-8125-4ab2-9b9c-b7b71fedbb5d None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:56.920629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4325b-8125-4ab2-9b9c-b7b71fedbb5d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:56.920980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4325b-8125-4ab2-9b9c-b7b71fedbb5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:56.922440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4325b-8125-4ab2-9b9c-b7b71fedbb5d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:56.928480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 475/952] 10.222.0.50 () {56 vars in 1122 bytes} [Fri Apr 29 18:30:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:30:56.938211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea700002-1353-4fe4-aa0d-da06277ce508 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:57.043756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea700002-1353-4fe4-aa0d-da06277ce508 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:57.076275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a145a-ec39-4ada-b078-6e049f319f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:57.077223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a145a-ec39-4ada-b078-6e049f319f00 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:57.077667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a145a-ec39-4ada-b078-6e049f319f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:57.078065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a145a-ec39-4ada-b078-6e049f319f00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:57.078988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a145a-ec39-4ada-b078-6e049f319f00 None admin] RBAC: Adding query filter params (interface=public, service_id=75a8a7d612f34c7f981398eba02c4ba6, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:57.079445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a145a-ec39-4ada-b078-6e049f319f00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:57.080980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a145a-ec39-4ada-b078-6e049f319f00 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:57.088601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 478/953] 10.222.0.50 () {56 vars in 1251 bytes} [Fri Apr 29 18:30:56 2022] GET /identity/v3/endpoints?service_id=75a8a7d612f34c7f981398eba02c4ba6&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:57.092998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4325b-8125-4ab2-9b9c-b7b71fedbb5d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:57.200701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4325b-8125-4ab2-9b9c-b7b71fedbb5d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:57.234824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021e57d-edbe-490d-b37f-0dc65514853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:57.235494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021e57d-edbe-490d-b37f-0dc65514853f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:57.235771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021e57d-edbe-490d-b37f-0dc65514853f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:57.236040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021e57d-edbe-490d-b37f-0dc65514853f None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:57.236750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021e57d-edbe-490d-b37f-0dc65514853f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:57.238337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021e57d-edbe-490d-b37f-0dc65514853f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:57.244862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 476/954] 10.222.0.50 () {56 vars in 1087 bytes} [Fri Apr 29 18:30:57 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:30:57.819923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd310-649e-49a5-87f8-9055c13fe079 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:57.820585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd310-649e-49a5-87f8-9055c13fe079 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:57.820668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd310-649e-49a5-87f8-9055c13fe079 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:57.822237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 479/955] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:57.827613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ce3a3-534e-4668-a8c0-4f70decc9263 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:57.828018 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ce3a3-534e-4668-a8c0-4f70decc9263 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:57.828182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ce3a3-534e-4668-a8c0-4f70decc9263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:57.845712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cd4ce3a3-534e-4668-a8c0-4f70decc9263 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:57.847189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cd4ce3a3-534e-4668-a8c0-4f70decc9263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:57.912253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 477/956] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:57.935325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fd310-649e-49a5-87f8-9055c13fe079 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:58.043840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fd310-649e-49a5-87f8-9055c13fe079 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:58.077172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80d4ff-6ff5-4db2-9086-73abe57f7e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:58.077899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80d4ff-6ff5-4db2-9086-73abe57f7e2f None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:58.078174 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80d4ff-6ff5-4db2-9086-73abe57f7e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:58.078440 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80d4ff-6ff5-4db2-9086-73abe57f7e2f None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:58.079229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80d4ff-6ff5-4db2-9086-73abe57f7e2f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:58.080610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80d4ff-6ff5-4db2-9086-73abe57f7e2f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:58.096862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 480/957] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:30:57 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:30:58.863166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9bb0c-b33b-40b0-9f08-ef8986875644 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:58.864146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9bb0c-b33b-40b0-9f08-ef8986875644 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:58.864663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9bb0c-b33b-40b0-9f08-ef8986875644 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:58.866469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 478/958] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:58.872210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78466710-93a8-4e79-8303-a54fad45031c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:58.873194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78466710-93a8-4e79-8303-a54fad45031c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:58.873687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78466710-93a8-4e79-8303-a54fad45031c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:58.891192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-78466710-93a8-4e79-8303-a54fad45031c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:58.892740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-78466710-93a8-4e79-8303-a54fad45031c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:58.972182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 481/959] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:58.997277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9bb0c-b33b-40b0-9f08-ef8986875644 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:59.098682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9bb0c-b33b-40b0-9f08-ef8986875644 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:59.127462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747b2f6-2b07-485d-a87a-8e7a563a96a6 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.128064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747b2f6-2b07-485d-a87a-8e7a563a96a6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.128567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747b2f6-2b07-485d-a87a-8e7a563a96a6 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.130807 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 482/960] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:30:59.138934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50769ce9-5229-499a-af3a-49e4ad5d91db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:59.140143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50769ce9-5229-499a-af3a-49e4ad5d91db None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.140143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50769ce9-5229-499a-af3a-49e4ad5d91db None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.140439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50769ce9-5229-499a-af3a-49e4ad5d91db None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.141142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50769ce9-5229-499a-af3a-49e4ad5d91db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:59.142187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240f77d-4a34-4641-8a66-3db9289a16a8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.142684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240f77d-4a34-4641-8a66-3db9289a16a8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.142788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50769ce9-5229-499a-af3a-49e4ad5d91db None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:59.143242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240f77d-4a34-4641-8a66-3db9289a16a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.157606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.sql.core [None req-50769ce9-5229-499a-af3a-49e4ad5d91db None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Apr 29 18:30:59.157606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Apr 29 18:30:59.157606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [parameters: {'user_id': '8eed257c28d241d3b0f96c3c48fe596b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Apr 29 18:30:59.157606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Apr 29 18:30:59.162959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1240f77d-4a34-4641-8a66-3db9289a16a8 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:59.164625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1240f77d-4a34-4641-8a66-3db9289a16a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-50769ce9-5229-499a-af3a-49e4ad5d91db None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Apr 29 18:30:59.165203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Apr 29 18:30:59.166064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application yield resource Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.rollback() Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.force_reraise() Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise self.value Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application session.commit() Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1431, in commit Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Apr 29 18:30:59.167046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._prepare_impl() Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.flush() Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3363, in flush Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._flush(objects) Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3503, in _flush Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application compat.raise_( Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3463, in _flush Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application flush_context.execute() Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rec.execute(self) Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Apr 29 18:30:59.167945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application _emit_insert_statements( Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = connection._execute_20( Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self._execute_context( Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Apr 29 18:30:59.168928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Apr 29 18:30:59.169862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [parameters: {'user_id': '8eed257c28d241d3b0f96c3c48fe596b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:59.170741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Apr 29 18:30:59.171569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Apr 29 18:30:59.174110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Apr 29 18:30:59.174110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Apr 29 18:30:59.174110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Apr 29 18:30:59.174110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Apr 29 18:30:59.174110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Apr 29 18:30:59.174110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:59.174110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 479/961] 10.222.0.50 () {60 vars in 1131 bytes} [Fri Apr 29 18:30:58 2022] POST /identity/v3/users => generated 160 bytes in 170 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:59.182323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50769ce9-5229-499a-af3a-49e4ad5d91db None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:59.248530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 483/962] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:30:59.267410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240f77d-4a34-4641-8a66-3db9289a16a8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:59.284483 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50769ce9-5229-499a-af3a-49e4ad5d91db None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:59.318264 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89080213-f98b-4e4d-bc6d-e8a0b3108ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:59.319063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89080213-f98b-4e4d-bc6d-e8a0b3108ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.319337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89080213-f98b-4e4d-bc6d-e8a0b3108ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.319600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89080213-f98b-4e4d-bc6d-e8a0b3108ef8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.327300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89080213-f98b-4e4d-bc6d-e8a0b3108ef8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:59.329043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89080213-f98b-4e4d-bc6d-e8a0b3108ef8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-89080213-f98b-4e4d-bc6d-e8a0b3108ef8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:30:59.343987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:59.344995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:59.345946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 480/963] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:30:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:59.351412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89080213-f98b-4e4d-bc6d-e8a0b3108ef8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:59.372145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240f77d-4a34-4641-8a66-3db9289a16a8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:59.409821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c20fcc-b701-4f63-8916-e182dcea3042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:59.411030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c20fcc-b701-4f63-8916-e182dcea3042 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.411511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c20fcc-b701-4f63-8916-e182dcea3042 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.411982 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c20fcc-b701-4f63-8916-e182dcea3042 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.422980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c20fcc-b701-4f63-8916-e182dcea3042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-62c20fcc-b701-4f63-8916-e182dcea3042 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:30:59.431756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:59.432743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:30:59.433637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:59.441444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 484/964] 10.222.0.50 () {56 vars in 1120 bytes} [Fri Apr 29 18:30:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:59.442561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c20fcc-b701-4f63-8916-e182dcea3042 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:59.454140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89080213-f98b-4e4d-bc6d-e8a0b3108ef8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:59.488020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7c824-6811-4d63-a859-d387e348453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:59.488730 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7c824-6811-4d63-a859-d387e348453a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.489008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7c824-6811-4d63-a859-d387e348453a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.489274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7c824-6811-4d63-a859-d387e348453a None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.490069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7c824-6811-4d63-a859-d387e348453a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:59.490366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7c824-6811-4d63-a859-d387e348453a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:59.491811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7c824-6811-4d63-a859-d387e348453a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:59.536785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 481/965] 10.222.0.50 () {56 vars in 1101 bytes} [Fri Apr 29 18:30:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:59.546352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c20fcc-b701-4f63-8916-e182dcea3042 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:59.581990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12cc71-86b0-4bf1-9cb7-a72e80a9cf1a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.582353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12cc71-86b0-4bf1-9cb7-a72e80a9cf1a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.582536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12cc71-86b0-4bf1-9cb7-a72e80a9cf1a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.584165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 482/966] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:30:59.585889 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc76fe-ae96-4511-9e26-bbb269c26178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:59.586577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc76fe-ae96-4511-9e26-bbb269c26178 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.586861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc76fe-ae96-4511-9e26-bbb269c26178 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.587137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc76fe-ae96-4511-9e26-bbb269c26178 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.587931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc76fe-ae96-4511-9e26-bbb269c26178 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:59.588246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc76fe-ae96-4511-9e26-bbb269c26178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:59.589667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc76fe-ae96-4511-9e26-bbb269c26178 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:59.595441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d1f6e-14e6-4d48-b6ba-de1f39c4aef6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.595441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d1f6e-14e6-4d48-b6ba-de1f39c4aef6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.595441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d1f6e-14e6-4d48-b6ba-de1f39c4aef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.604552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 485/967] 10.222.0.50 () {56 vars in 1115 bytes} [Fri Apr 29 18:30:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:30:59.606367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc76fe-ae96-4511-9e26-bbb269c26178 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:59.619140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f87d1f6e-14e6-4d48-b6ba-de1f39c4aef6 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:30:59.623450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f87d1f6e-14e6-4d48-b6ba-de1f39c4aef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:30:59.695917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 483/968] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:30:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:30:59.718608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d1f6e-14e6-4d48-b6ba-de1f39c4aef6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:59.738268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc76fe-ae96-4511-9e26-bbb269c26178 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:59.794488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f646f-25e0-4e6c-acb6-0e56a2b2bf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:59.795397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f646f-25e0-4e6c-acb6-0e56a2b2bf5b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.795763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f646f-25e0-4e6c-acb6-0e56a2b2bf5b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.796087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f646f-25e0-4e6c-acb6-0e56a2b2bf5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.807962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f646f-25e0-4e6c-acb6-0e56a2b2bf5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:59.810366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f646f-25e0-4e6c-acb6-0e56a2b2bf5b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-dd0f646f-25e0-4e6c-acb6-0e56a2b2bf5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:30:59.829785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:59.830742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:59.831639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:30:59.831639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:30:59.831639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:30:59.831639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:30:59.831639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 486/969] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:30:59.837280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f646f-25e0-4e6c-acb6-0e56a2b2bf5b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:59.857849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d1f6e-14e6-4d48-b6ba-de1f39c4aef6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:59.894598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9725288-a0fe-4cec-8c65-cf4cc5c25ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:59.895412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9725288-a0fe-4cec-8c65-cf4cc5c25ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.895713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9725288-a0fe-4cec-8c65-cf4cc5c25ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.895949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9725288-a0fe-4cec-8c65-cf4cc5c25ca3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.896626 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9725288-a0fe-4cec-8c65-cf4cc5c25ca3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:59.897993 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9725288-a0fe-4cec-8c65-cf4cc5c25ca3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b9725288-a0fe-4cec-8c65-cf4cc5c25ca3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:30:59.909930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:30:59.911078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:30:59.911987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:30:59.916199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 484/970] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:30:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:30:59.917056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9725288-a0fe-4cec-8c65-cf4cc5c25ca3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:30:59.949052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f646f-25e0-4e6c-acb6-0e56a2b2bf5b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:30:59.983234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb5147-a4ad-4de5-9b99-aceedaf12bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:30:59.983899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb5147-a4ad-4de5-9b99-aceedaf12bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:30:59.984177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb5147-a4ad-4de5-9b99-aceedaf12bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:30:59.984449 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb5147-a4ad-4de5-9b99-aceedaf12bb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:30:59.985205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb5147-a4ad-4de5-9b99-aceedaf12bb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:30:59.985508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb5147-a4ad-4de5-9b99-aceedaf12bb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:30:59.986872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb5147-a4ad-4de5-9b99-aceedaf12bb2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.003497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 487/971] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:00.004576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb5147-a4ad-4de5-9b99-aceedaf12bb2 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.012166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9725288-a0fe-4cec-8c65-cf4cc5c25ca3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.046802 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09554af0-8b0b-43bb-82be-0310960cf181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.047492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09554af0-8b0b-43bb-82be-0310960cf181 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.047767 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09554af0-8b0b-43bb-82be-0310960cf181 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.048036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09554af0-8b0b-43bb-82be-0310960cf181 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.048779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09554af0-8b0b-43bb-82be-0310960cf181 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:00.049083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09554af0-8b0b-43bb-82be-0310960cf181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.050463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09554af0-8b0b-43bb-82be-0310960cf181 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.056339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 485/972] 10.222.0.50 () {56 vars in 1122 bytes} [Fri Apr 29 18:30:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:00.062035 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09554af0-8b0b-43bb-82be-0310960cf181 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.104766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb5147-a4ad-4de5-9b99-aceedaf12bb2 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.141231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0271583-5349-4bfc-957d-3124f72387f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.142069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0271583-5349-4bfc-957d-3124f72387f4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.142339 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0271583-5349-4bfc-957d-3124f72387f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.142618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0271583-5349-4bfc-957d-3124f72387f4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.150097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0271583-5349-4bfc-957d-3124f72387f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.151771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0271583-5349-4bfc-957d-3124f72387f4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.164560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09554af0-8b0b-43bb-82be-0310960cf181 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f0271583-5349-4bfc-957d-3124f72387f4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:31:00.166498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:00.167433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:00.168378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 488/973] 10.222.0.50 () {56 vars in 1117 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:31:00.173527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0271583-5349-4bfc-957d-3124f72387f4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.202864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa45d16-1733-47fa-88b0-1801a9d33fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.203578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa45d16-1733-47fa-88b0-1801a9d33fbc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.203862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa45d16-1733-47fa-88b0-1801a9d33fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.204133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa45d16-1733-47fa-88b0-1801a9d33fbc None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.204891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa45d16-1733-47fa-88b0-1801a9d33fbc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:00.205195 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa45d16-1733-47fa-88b0-1801a9d33fbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.206604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa45d16-1733-47fa-88b0-1801a9d33fbc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.212623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 486/974] 10.222.0.50 () {56 vars in 1122 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:00.218751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa45d16-1733-47fa-88b0-1801a9d33fbc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.276908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0271583-5349-4bfc-957d-3124f72387f4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.311440 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c38eb4-d741-42de-8789-7b78417cd1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.312118 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c38eb4-d741-42de-8789-7b78417cd1ab None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.312394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c38eb4-d741-42de-8789-7b78417cd1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.312689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c38eb4-d741-42de-8789-7b78417cd1ab None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.313513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c38eb4-d741-42de-8789-7b78417cd1ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:00.313814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c38eb4-d741-42de-8789-7b78417cd1ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.315233 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c38eb4-d741-42de-8789-7b78417cd1ab None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.315920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa45d16-1733-47fa-88b0-1801a9d33fbc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.337463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 489/975] 10.222.0.50 () {56 vars in 1149 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:00.338957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c38eb4-d741-42de-8789-7b78417cd1ab None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.350776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf9880-21fb-4c19-8ba4-922cc281bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.351721 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf9880-21fb-4c19-8ba4-922cc281bfca None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.352015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf9880-21fb-4c19-8ba4-922cc281bfca None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.352288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf9880-21fb-4c19-8ba4-922cc281bfca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.353067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf9880-21fb-4c19-8ba4-922cc281bfca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.354429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf9880-21fb-4c19-8ba4-922cc281bfca None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.373567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 487/976] 10.222.0.50 () {60 vars in 1144 bytes} [Fri Apr 29 18:31:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:31:00.384717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144b463-97d2-441c-a323-fc87b5096129 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.384717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144b463-97d2-441c-a323-fc87b5096129 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.384717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144b463-97d2-441c-a323-fc87b5096129 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.385515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 488/977] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:00.392098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d88955-0d87-43ac-b0ab-9219c10a288e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.392410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d88955-0d87-43ac-b0ab-9219c10a288e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.392690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d88955-0d87-43ac-b0ab-9219c10a288e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.417375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d6d88955-0d87-43ac-b0ab-9219c10a288e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:00.418649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6d88955-0d87-43ac-b0ab-9219c10a288e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:00.460834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c38eb4-d741-42de-8789-7b78417cd1ab None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.500528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 489/978] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:31:00.503232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8003dd-97fe-4e45-801c-74f8ee35d516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.504023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8003dd-97fe-4e45-801c-74f8ee35d516 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.504397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8003dd-97fe-4e45-801c-74f8ee35d516 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.504801 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8003dd-97fe-4e45-801c-74f8ee35d516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.523091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d88955-0d87-43ac-b0ab-9219c10a288e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.577941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8003dd-97fe-4e45-801c-74f8ee35d516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.579397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8003dd-97fe-4e45-801c-74f8ee35d516 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.625548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d88955-0d87-43ac-b0ab-9219c10a288e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.657710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9700de-3db0-4200-8a51-abee01670940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.658603 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9700de-3db0-4200-8a51-abee01670940 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.658946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9700de-3db0-4200-8a51-abee01670940 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.659311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9700de-3db0-4200-8a51-abee01670940 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.660056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9700de-3db0-4200-8a51-abee01670940 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.661542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9700de-3db0-4200-8a51-abee01670940 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ab9700de-3db0-4200-8a51-abee01670940 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:00.673027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:00.674241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:00.675104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 490/979] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:31:00.679678 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 490/980] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:00.680226 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9700de-3db0-4200-8a51-abee01670940 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.685387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8003dd-97fe-4e45-801c-74f8ee35d516 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.777801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9700de-3db0-4200-8a51-abee01670940 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.780093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8003dd-97fe-4e45-801c-74f8ee35d516 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.810076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8f1f1-5616-4b35-a842-1a3f1ebd174e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.810821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8f1f1-5616-4b35-a842-1a3f1ebd174e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.811177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8f1f1-5616-4b35-a842-1a3f1ebd174e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.811499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8f1f1-5616-4b35-a842-1a3f1ebd174e None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.812275 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8f1f1-5616-4b35-a842-1a3f1ebd174e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:00.812665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8f1f1-5616-4b35-a842-1a3f1ebd174e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.813326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0210a-b819-414e-af48-c80685acd515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.814151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8f1f1-5616-4b35-a842-1a3f1ebd174e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.814247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0210a-b819-414e-af48-c80685acd515 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.814567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0210a-b819-414e-af48-c80685acd515 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.814906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0210a-b819-414e-af48-c80685acd515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.819265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 491/981] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:00.824733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8f1f1-5616-4b35-a842-1a3f1ebd174e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.825386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0210a-b819-414e-af48-c80685acd515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.826830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0210a-b819-414e-af48-c80685acd515 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-8ee0210a-b819-414e-af48-c80685acd515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:31:00.844155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:00.845377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:00.846306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:31:00.846306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:31:00.846306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:00.846306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:00.846306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 491/982] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:00.850865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0210a-b819-414e-af48-c80685acd515 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.926333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8f1f1-5616-4b35-a842-1a3f1ebd174e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.945593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0210a-b819-414e-af48-c80685acd515 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:00.957734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af98fe2-0163-490d-bb88-3c7e07fdae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.958397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af98fe2-0163-490d-bb88-3c7e07fdae66 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.958672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af98fe2-0163-490d-bb88-3c7e07fdae66 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.958940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af98fe2-0163-490d-bb88-3c7e07fdae66 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.959633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af98fe2-0163-490d-bb88-3c7e07fdae66 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:00.959935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af98fe2-0163-490d-bb88-3c7e07fdae66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.961466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af98fe2-0163-490d-bb88-3c7e07fdae66 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.966370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 492/983] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:00.972411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af98fe2-0163-490d-bb88-3c7e07fdae66 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:00.978024 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b097-e1f7-4e8c-bc66-65902996f333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:00.978734 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b097-e1f7-4e8c-bc66-65902996f333 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:00.979017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b097-e1f7-4e8c-bc66-65902996f333 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:00.979265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b097-e1f7-4e8c-bc66-65902996f333 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:00.979992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200b097-e1f7-4e8c-bc66-65902996f333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:00.980290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200b097-e1f7-4e8c-bc66-65902996f333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:00.981674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200b097-e1f7-4e8c-bc66-65902996f333 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:00.997831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 492/984] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:00.998830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b097-e1f7-4e8c-bc66-65902996f333 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:01.067293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af98fe2-0163-490d-bb88-3c7e07fdae66 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:01.094003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b097-e1f7-4e8c-bc66-65902996f333 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:01.099462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f9565-13ad-458c-bfdb-c17ace153f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:01.100127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f9565-13ad-458c-bfdb-c17ace153f2e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:01.100404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f9565-13ad-458c-bfdb-c17ace153f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:01.100717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f9565-13ad-458c-bfdb-c17ace153f2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:01.101557 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f9565-13ad-458c-bfdb-c17ace153f2e None admin] RBAC: Adding query filter params (interface=public, service_id=8e4f5b61961c4c5499debee4e0828d27, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:01.101845 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f9565-13ad-458c-bfdb-c17ace153f2e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:01.103205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f9565-13ad-458c-bfdb-c17ace153f2e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:01.112828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 493/985] 10.222.0.50 () {56 vars in 1251 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/endpoints?service_id=8e4f5b61961c4c5499debee4e0828d27&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:01.113666 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f9565-13ad-458c-bfdb-c17ace153f2e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:01.127583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2446e-94eb-41cc-83e8-a6833fcb8536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:01.128653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2446e-94eb-41cc-83e8-a6833fcb8536 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:01.129095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2446e-94eb-41cc-83e8-a6833fcb8536 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:01.129355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2446e-94eb-41cc-83e8-a6833fcb8536 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:01.136416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2446e-94eb-41cc-83e8-a6833fcb8536 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:01.137891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2446e-94eb-41cc-83e8-a6833fcb8536 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5fa2446e-94eb-41cc-83e8-a6833fcb8536 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:31:01.151633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:01.152724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:01.153633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:31:01.153633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:31:01.153633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:31:01.153633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:31:01.153633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:31:01.153633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:31:01.153633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:31:01.153633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:01.153633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 493/986] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:01.158525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2446e-94eb-41cc-83e8-a6833fcb8536 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:01.213844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f9565-13ad-458c-bfdb-c17ace153f2e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:01.252759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf1ada-2e94-415e-bbdd-9b2f36961213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:01.254247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf1ada-2e94-415e-bbdd-9b2f36961213 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:01.254671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf1ada-2e94-415e-bbdd-9b2f36961213 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:01.255236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf1ada-2e94-415e-bbdd-9b2f36961213 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:01.256338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf1ada-2e94-415e-bbdd-9b2f36961213 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:01.258173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf1ada-2e94-415e-bbdd-9b2f36961213 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:01.258634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2446e-94eb-41cc-83e8-a6833fcb8536 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:01.266285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 494/987] 10.222.0.50 () {56 vars in 1087 bytes} [Fri Apr 29 18:31:01 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:31:01.292150 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85fc06-1034-4842-a170-7d1c8676c319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:01.292894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85fc06-1034-4842-a170-7d1c8676c319 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:01.293171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85fc06-1034-4842-a170-7d1c8676c319 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:01.293437 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85fc06-1034-4842-a170-7d1c8676c319 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:01.294242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85fc06-1034-4842-a170-7d1c8676c319 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:01.294542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85fc06-1034-4842-a170-7d1c8676c319 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:01.296023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85fc06-1034-4842-a170-7d1c8676c319 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:01.315265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 494/988] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:01.316707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf1ada-2e94-415e-bbdd-9b2f36961213 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:01.422544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf1ada-2e94-415e-bbdd-9b2f36961213 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:01.458597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e0103-9088-4116-98ba-442634b3aff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:01.458938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e0103-9088-4116-98ba-442634b3aff0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:01.459192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e0103-9088-4116-98ba-442634b3aff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:01.459461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e0103-9088-4116-98ba-442634b3aff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:01.460269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e0103-9088-4116-98ba-442634b3aff0 None admin] RBAC: Adding query filter params (role.id=6f04dd8fb0cf4f47a384d0fcca2217ea, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=b7edbbc857d942d685c124c2c6aac09e) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:01.460680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e0103-9088-4116-98ba-442634b3aff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:01.462157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e0103-9088-4116-98ba-442634b3aff0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:01.472686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 495/989] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:31:01 2022] GET /identity/v3/role_assignments?user.id=b7edbbc857d942d685c124c2c6aac09e&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=6f04dd8fb0cf4f47a384d0fcca2217ea => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:02.691220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a31fe-8789-4175-9800-2efca4687e20 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:02.691759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a31fe-8789-4175-9800-2efca4687e20 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:02.691759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a31fe-8789-4175-9800-2efca4687e20 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:02.693128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 495/990] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:02.699135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d89af4-3048-48aa-b8d8-a8a5d0c23ce1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:02.699551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d89af4-3048-48aa-b8d8-a8a5d0c23ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:02.699735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d89af4-3048-48aa-b8d8-a8a5d0c23ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:02.717603 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a8d89af4-3048-48aa-b8d8-a8a5d0c23ce1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:02.718777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a8d89af4-3048-48aa-b8d8-a8a5d0c23ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:02.788531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 496/991] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:02.808456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a31fe-8789-4175-9800-2efca4687e20 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:02.909774 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a31fe-8789-4175-9800-2efca4687e20 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:02.943991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927e7d5-63d3-459f-b706-66956d9a4514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:02.944843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927e7d5-63d3-459f-b706-66956d9a4514 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:02.945188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927e7d5-63d3-459f-b706-66956d9a4514 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:02.945505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927e7d5-63d3-459f-b706-66956d9a4514 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:02.946160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927e7d5-63d3-459f-b706-66956d9a4514 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:02.947538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927e7d5-63d3-459f-b706-66956d9a4514 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0927e7d5-63d3-459f-b706-66956d9a4514 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:02.959673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:02.961022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:31:02.962263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:02.964875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 496/992] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:31:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:02.967354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d89af4-3048-48aa-b8d8-a8a5d0c23ce1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:03.065037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d89af4-3048-48aa-b8d8-a8a5d0c23ce1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:03.100878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da26754-67b3-44a1-b90a-877e7ccd3095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:03.101692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da26754-67b3-44a1-b90a-877e7ccd3095 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.101965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da26754-67b3-44a1-b90a-877e7ccd3095 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.102286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da26754-67b3-44a1-b90a-877e7ccd3095 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.102989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da26754-67b3-44a1-b90a-877e7ccd3095 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:03.103286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da26754-67b3-44a1-b90a-877e7ccd3095 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:03.104670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da26754-67b3-44a1-b90a-877e7ccd3095 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:03.114112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 497/993] 10.222.0.50 () {56 vars in 1122 bytes} [Fri Apr 29 18:31:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:03.115343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927e7d5-63d3-459f-b706-66956d9a4514 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:03.212208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927e7d5-63d3-459f-b706-66956d9a4514 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:03.246562 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc6dc2-16cc-4be6-b348-32faaf93e09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:03.247231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc6dc2-16cc-4be6-b348-32faaf93e09b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.247509 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc6dc2-16cc-4be6-b348-32faaf93e09b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.247779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc6dc2-16cc-4be6-b348-32faaf93e09b None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.248511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc6dc2-16cc-4be6-b348-32faaf93e09b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:03.248828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc6dc2-16cc-4be6-b348-32faaf93e09b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:03.250210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc6dc2-16cc-4be6-b348-32faaf93e09b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:03.255167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 497/994] 10.222.0.50 () {56 vars in 1122 bytes} [Fri Apr 29 18:31:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:03.261108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da26754-67b3-44a1-b90a-877e7ccd3095 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:03.361386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da26754-67b3-44a1-b90a-877e7ccd3095 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:03.396636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56438b-203a-46df-9a93-6aa4fa88a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:03.397301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56438b-203a-46df-9a93-6aa4fa88a82a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.397569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56438b-203a-46df-9a93-6aa4fa88a82a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.397848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56438b-203a-46df-9a93-6aa4fa88a82a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.398760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56438b-203a-46df-9a93-6aa4fa88a82a None admin] RBAC: Adding query filter params (interface=admin, service_id=75a8a7d612f34c7f981398eba02c4ba6, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:03.399060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56438b-203a-46df-9a93-6aa4fa88a82a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:03.400428 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56438b-203a-46df-9a93-6aa4fa88a82a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:03.405683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 498/995] 10.222.0.50 () {56 vars in 1249 bytes} [Fri Apr 29 18:31:03 2022] GET /identity/v3/endpoints?service_id=75a8a7d612f34c7f981398eba02c4ba6&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:03.411088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc6dc2-16cc-4be6-b348-32faaf93e09b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:03.506844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc6dc2-16cc-4be6-b348-32faaf93e09b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:03.528270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4918ff3-2be3-4aaa-8867-f18d556d336c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.528609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4918ff3-2be3-4aaa-8867-f18d556d336c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.528855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4918ff3-2be3-4aaa-8867-f18d556d336c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.530248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 499/996] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:03.535332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075c0ab-0ed8-4322-aa52-64016b7bf315 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.535608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075c0ab-0ed8-4322-aa52-64016b7bf315 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.535896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075c0ab-0ed8-4322-aa52-64016b7bf315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.539177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd30d6-fcc6-46af-abad-9d5937fe325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:03.540079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd30d6-fcc6-46af-abad-9d5937fe325c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.540368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd30d6-fcc6-46af-abad-9d5937fe325c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.540681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd30d6-fcc6-46af-abad-9d5937fe325c None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.541371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd30d6-fcc6-46af-abad-9d5937fe325c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:03.542799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd30d6-fcc6-46af-abad-9d5937fe325c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:03.549695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 498/997] 10.222.0.50 () {56 vars in 1087 bytes} [Fri Apr 29 18:31:03 2022] GET /identity/v3/services => generated 1727 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:31:03.554791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4075c0ab-0ed8-4322-aa52-64016b7bf315 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:03.555757 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4075c0ab-0ed8-4322-aa52-64016b7bf315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:03.637238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 500/998] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:31:03.660804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd30d6-fcc6-46af-abad-9d5937fe325c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:03.747971 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69352dd-f476-4930-ba52-32bcaf37ecae None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.748594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69352dd-f476-4930-ba52-32bcaf37ecae None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.748594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69352dd-f476-4930-ba52-32bcaf37ecae None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.750045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 501/999] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:31:03.754824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7eee0-0325-4573-8e30-ca3cc404d8d1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.755504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7eee0-0325-4573-8e30-ca3cc404d8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.755504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7eee0-0325-4573-8e30-ca3cc404d8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.769273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd30d6-fcc6-46af-abad-9d5937fe325c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:03.773708 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f5d7eee0-0325-4573-8e30-ca3cc404d8d1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:03.774900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f5d7eee0-0325-4573-8e30-ca3cc404d8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:03.801987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003034f7-9fd7-40a8-86df-19647a72992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:03.802790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003034f7-9fd7-40a8-86df-19647a72992f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.803065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003034f7-9fd7-40a8-86df-19647a72992f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.803333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003034f7-9fd7-40a8-86df-19647a72992f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.803978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003034f7-9fd7-40a8-86df-19647a72992f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:03.805624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003034f7-9fd7-40a8-86df-19647a72992f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-003034f7-9fd7-40a8-86df-19647a72992f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:03.817013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:03.818036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:03.818911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 499/1000] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:31:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:03.823977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003034f7-9fd7-40a8-86df-19647a72992f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:03.843324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 502/1001] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:03.864718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7eee0-0325-4573-8e30-ca3cc404d8d1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:03.923335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003034f7-9fd7-40a8-86df-19647a72992f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:03.956049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a6bda-4aff-4ce1-9771-568b340595bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:03.956821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a6bda-4aff-4ce1-9771-568b340595bc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.957103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a6bda-4aff-4ce1-9771-568b340595bc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.957203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7eee0-0325-4573-8e30-ca3cc404d8d1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:03.957373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a6bda-4aff-4ce1-9771-568b340595bc None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.958108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a6bda-4aff-4ce1-9771-568b340595bc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:03.958383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a6bda-4aff-4ce1-9771-568b340595bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:03.959753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a6bda-4aff-4ce1-9771-568b340595bc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:03.969291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 500/1002] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:31:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:03.970950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a6bda-4aff-4ce1-9771-568b340595bc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:03.990963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101b615-9e48-41db-9149-bf536cc0c1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:03.991757 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101b615-9e48-41db-9149-bf536cc0c1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:03.992151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101b615-9e48-41db-9149-bf536cc0c1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:03.992425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101b615-9e48-41db-9149-bf536cc0c1a1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:03.993103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101b615-9e48-41db-9149-bf536cc0c1a1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:03.994461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101b615-9e48-41db-9149-bf536cc0c1a1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7101b615-9e48-41db-9149-bf536cc0c1a1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:04.005541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:04.006533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:04.013009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 503/1003] 10.222.0.50 () {56 vars in 1105 bytes} [Fri Apr 29 18:31:03 2022] GET /identity/v3/services/image => generated 86 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:04.013814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101b615-9e48-41db-9149-bf536cc0c1a1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:04.073573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a6bda-4aff-4ce1-9771-568b340595bc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:04.109065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101b615-9e48-41db-9149-bf536cc0c1a1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:04.112688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bf627-0813-492d-ba54-587838c8b751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:04.113358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bf627-0813-492d-ba54-587838c8b751 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:04.113638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bf627-0813-492d-ba54-587838c8b751 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:04.113909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bf627-0813-492d-ba54-587838c8b751 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:04.114609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bf627-0813-492d-ba54-587838c8b751 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:04.114910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bf627-0813-492d-ba54-587838c8b751 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:04.116723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bf627-0813-492d-ba54-587838c8b751 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:04.123591 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 501/1004] 10.222.0.50 () {56 vars in 1112 bytes} [Fri Apr 29 18:31:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:04.129111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bf627-0813-492d-ba54-587838c8b751 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:04.142374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09151310-3e58-409d-a7aa-64b0c756eaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:04.143141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151310-3e58-409d-a7aa-64b0c756eaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:04.143482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151310-3e58-409d-a7aa-64b0c756eaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:04.143812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151310-3e58-409d-a7aa-64b0c756eaa8 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:04.144609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09151310-3e58-409d-a7aa-64b0c756eaa8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:04.144979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09151310-3e58-409d-a7aa-64b0c756eaa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:04.146427 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09151310-3e58-409d-a7aa-64b0c756eaa8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:04.156298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 504/1005] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:04.157339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09151310-3e58-409d-a7aa-64b0c756eaa8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:04.238250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bf627-0813-492d-ba54-587838c8b751 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:04.255127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09151310-3e58-409d-a7aa-64b0c756eaa8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:04.270938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2fa2f-da2d-4374-ac32-d037f1f29143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:04.271591 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2fa2f-da2d-4374-ac32-d037f1f29143 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:04.271889 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2fa2f-da2d-4374-ac32-d037f1f29143 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:04.272157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2fa2f-da2d-4374-ac32-d037f1f29143 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:04.272935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2fa2f-da2d-4374-ac32-d037f1f29143 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:04.274440 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2fa2f-da2d-4374-ac32-d037f1f29143 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:04.288567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c99e5-fcf9-43c3-8eb9-9dde3e8eccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:04.289332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c99e5-fcf9-43c3-8eb9-9dde3e8eccd3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:04.289601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c99e5-fcf9-43c3-8eb9-9dde3e8eccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:04.289922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c99e5-fcf9-43c3-8eb9-9dde3e8eccd3 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:04.290640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c99e5-fcf9-43c3-8eb9-9dde3e8eccd3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:04.290963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c99e5-fcf9-43c3-8eb9-9dde3e8eccd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:04.292314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c99e5-fcf9-43c3-8eb9-9dde3e8eccd3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:04.304860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 505/1006] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:04.305094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 502/1007] 10.222.0.50 () {60 vars in 1144 bytes} [Fri Apr 29 18:31:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:31:05.811615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56551-9fdc-4954-8cd2-1f075d69df83 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:05.812179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56551-9fdc-4954-8cd2-1f075d69df83 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:05.812400 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56551-9fdc-4954-8cd2-1f075d69df83 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:05.814539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 506/1008] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:05.820997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb05c5-0eda-4fea-b93f-d37c84b6a5ba None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:05.821460 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb05c5-0eda-4fea-b93f-d37c84b6a5ba None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:05.821972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb05c5-0eda-4fea-b93f-d37c84b6a5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:05.847404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c9eb05c5-0eda-4fea-b93f-d37c84b6a5ba None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:05.848606 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c9eb05c5-0eda-4fea-b93f-d37c84b6a5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:05.919057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 503/1009] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:05 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:05.942172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56551-9fdc-4954-8cd2-1f075d69df83 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:06.042807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56551-9fdc-4954-8cd2-1f075d69df83 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:06.078773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef5f9b-a783-47dd-9016-59a46cb9c3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:06.079582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef5f9b-a783-47dd-9016-59a46cb9c3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:06.079862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef5f9b-a783-47dd-9016-59a46cb9c3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:06.080132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef5f9b-a783-47dd-9016-59a46cb9c3d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:06.080810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef5f9b-a783-47dd-9016-59a46cb9c3d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:06.082185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef5f9b-a783-47dd-9016-59a46cb9c3d7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a1ef5f9b-a783-47dd-9016-59a46cb9c3d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:06.094677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:06.095665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Apr 29 18:31:06.096535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:06.100970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 507/1010] 10.222.0.50 () {56 vars in 1126 bytes} [Fri Apr 29 18:31:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:06.102212 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb05c5-0eda-4fea-b93f-d37c84b6a5ba None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:06.203157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb05c5-0eda-4fea-b93f-d37c84b6a5ba None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:06.237098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a4578-f5a5-488d-b174-d1eb0706028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:06.237760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4578-f5a5-488d-b174-d1eb0706028c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:06.238014 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4578-f5a5-488d-b174-d1eb0706028c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:06.238304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4578-f5a5-488d-b174-d1eb0706028c None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:06.238997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a4578-f5a5-488d-b174-d1eb0706028c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:06.239295 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a4578-f5a5-488d-b174-d1eb0706028c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:06.240685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a4578-f5a5-488d-b174-d1eb0706028c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:06.250101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 504/1011] 10.222.0.50 () {56 vars in 1122 bytes} [Fri Apr 29 18:31:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:06.251367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef5f9b-a783-47dd-9016-59a46cb9c3d7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:06.347978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef5f9b-a783-47dd-9016-59a46cb9c3d7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:06.384349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb18302-b7e0-4871-90f9-975803cfea0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:06.385062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb18302-b7e0-4871-90f9-975803cfea0e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:06.385341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb18302-b7e0-4871-90f9-975803cfea0e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:06.385598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb18302-b7e0-4871-90f9-975803cfea0e None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:06.386306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb18302-b7e0-4871-90f9-975803cfea0e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:06.386600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb18302-b7e0-4871-90f9-975803cfea0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:06.387958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb18302-b7e0-4871-90f9-975803cfea0e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:06.397611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 508/1012] 10.222.0.50 () {56 vars in 1122 bytes} [Fri Apr 29 18:31:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:06.399569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a4578-f5a5-488d-b174-d1eb0706028c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:06.501665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a4578-f5a5-488d-b174-d1eb0706028c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:06.539878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478eaee-ba3b-4dbe-bf14-9bea7e4b1351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:06.540560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478eaee-ba3b-4dbe-bf14-9bea7e4b1351 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:06.540843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478eaee-ba3b-4dbe-bf14-9bea7e4b1351 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:06.541114 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478eaee-ba3b-4dbe-bf14-9bea7e4b1351 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:06.541866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478eaee-ba3b-4dbe-bf14-9bea7e4b1351 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:06.541961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce9754-882b-4f8c-91dc-8fa36501d45f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:06.542178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce9754-882b-4f8c-91dc-8fa36501d45f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:06.542452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce9754-882b-4f8c-91dc-8fa36501d45f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:06.543236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478eaee-ba3b-4dbe-bf14-9bea7e4b1351 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:06.543984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 509/1013] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:06.549022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f28d01-53d7-4767-9ad4-081b5a2b8dfa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:06.549312 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f28d01-53d7-4767-9ad4-081b5a2b8dfa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:06.549581 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f28d01-53d7-4767-9ad4-081b5a2b8dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:06.564523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 505/1014] 10.222.0.50 () {60 vars in 1144 bytes} [Fri Apr 29 18:31:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:31:06.566303 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c0f28d01-53d7-4767-9ad4-081b5a2b8dfa None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:06.567501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c0f28d01-53d7-4767-9ad4-081b5a2b8dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:06.656943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 510/1015] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:06 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:31:06.681062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478eaee-ba3b-4dbe-bf14-9bea7e4b1351 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:06.681321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996b4e-3939-4bf5-8f62-77b17b433823 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:06.681536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996b4e-3939-4bf5-8f62-77b17b433823 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:06.681808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0996b4e-3939-4bf5-8f62-77b17b433823 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:06.683403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 511/1016] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:06.688788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d4feb-500e-41d8-b816-55e3fdab8c3d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:06.689119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d4feb-500e-41d8-b816-55e3fdab8c3d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:06.689295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d4feb-500e-41d8-b816-55e3fdab8c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:06.707114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-925d4feb-500e-41d8-b816-55e3fdab8c3d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:06.708370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-925d4feb-500e-41d8-b816-55e3fdab8c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:06.781791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 512/1017] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:06 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:31:06.786236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478eaee-ba3b-4dbe-bf14-9bea7e4b1351 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:06.805345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d4feb-500e-41d8-b816-55e3fdab8c3d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:06.820607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841830a-f7c7-4ea3-8b97-238cd27cf7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:06.821277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841830a-f7c7-4ea3-8b97-238cd27cf7d0 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:06.821830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841830a-f7c7-4ea3-8b97-238cd27cf7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:06.821830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841830a-f7c7-4ea3-8b97-238cd27cf7d0 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:06.822567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841830a-f7c7-4ea3-8b97-238cd27cf7d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:06.823899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841830a-f7c7-4ea3-8b97-238cd27cf7d0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:06.838675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 506/1018] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:31:06 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:31:06.915474 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d4feb-500e-41d8-b816-55e3fdab8c3d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:06.960364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d276e4e-be3b-407b-9f97-98e912813a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:06.962019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d276e4e-be3b-407b-9f97-98e912813a0f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:06.962298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d276e4e-be3b-407b-9f97-98e912813a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:06.962570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d276e4e-be3b-407b-9f97-98e912813a0f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:06.974622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d276e4e-be3b-407b-9f97-98e912813a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3d276e4e-be3b-407b-9f97-98e912813a0f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:31:06.983785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:06.984901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:06.985764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 513/1019] 10.222.0.50 () {56 vars in 1120 bytes} [Fri Apr 29 18:31:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:31:06.991254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841830a-f7c7-4ea3-8b97-238cd27cf7d0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:07.094634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841830a-f7c7-4ea3-8b97-238cd27cf7d0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:07.127809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef925e6-d867-4ea2-84e4-6c84504a784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:07.128507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef925e6-d867-4ea2-84e4-6c84504a784c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:07.128790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef925e6-d867-4ea2-84e4-6c84504a784c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:07.129063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef925e6-d867-4ea2-84e4-6c84504a784c None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:07.129799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef925e6-d867-4ea2-84e4-6c84504a784c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:07.130096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef925e6-d867-4ea2-84e4-6c84504a784c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:07.131461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef925e6-d867-4ea2-84e4-6c84504a784c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:07.139957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 507/1020] 10.222.0.50 () {56 vars in 1115 bytes} [Fri Apr 29 18:31:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:07.145504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d276e4e-be3b-407b-9f97-98e912813a0f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:07.240998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d276e4e-be3b-407b-9f97-98e912813a0f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:07.272916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e8ba7-557f-4ed8-be20-b401e8a19a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:07.273803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e8ba7-557f-4ed8-be20-b401e8a19a1a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:07.274077 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e8ba7-557f-4ed8-be20-b401e8a19a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:07.274330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e8ba7-557f-4ed8-be20-b401e8a19a1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:07.284541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e8ba7-557f-4ed8-be20-b401e8a19a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:07.285972 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e8ba7-557f-4ed8-be20-b401e8a19a1a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a19e8ba7-557f-4ed8-be20-b401e8a19a1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:31:07.302776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:07.303716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:07.305135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:31:07.305135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:31:07.305135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:07.305135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:07.305709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 514/1021] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:07.310012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef925e6-d867-4ea2-84e4-6c84504a784c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:07.410138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef925e6-d867-4ea2-84e4-6c84504a784c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:07.452063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22d52e-4b93-4ab8-888e-996e8d08249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:07.452770 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d52e-4b93-4ab8-888e-996e8d08249f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:07.453055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d52e-4b93-4ab8-888e-996e8d08249f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:07.453329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d52e-4b93-4ab8-888e-996e8d08249f None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:07.454046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22d52e-4b93-4ab8-888e-996e8d08249f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:07.454349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22d52e-4b93-4ab8-888e-996e8d08249f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:07.455723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22d52e-4b93-4ab8-888e-996e8d08249f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:07.472712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 508/1022] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:07.474622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e8ba7-557f-4ed8-be20-b401e8a19a1a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:07.574354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e8ba7-557f-4ed8-be20-b401e8a19a1a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:07.613408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b56052-3fd5-4e2c-a482-a648fb4ac9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:07.614208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b56052-3fd5-4e2c-a482-a648fb4ac9ce None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:07.614489 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b56052-3fd5-4e2c-a482-a648fb4ac9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:07.614738 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b56052-3fd5-4e2c-a482-a648fb4ac9ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:07.623680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b56052-3fd5-4e2c-a482-a648fb4ac9ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:07.625170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b56052-3fd5-4e2c-a482-a648fb4ac9ce None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-70b56052-3fd5-4e2c-a482-a648fb4ac9ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:31:07.640139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:07.641188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:07.642164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 515/1023] 10.222.0.50 () {56 vars in 1093 bytes} [Fri Apr 29 18:31:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:07.648800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22d52e-4b93-4ab8-888e-996e8d08249f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:07.764914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22d52e-4b93-4ab8-888e-996e8d08249f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:07.798905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec739-239a-4b5a-89e4-2bb89ddbb1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:07.799577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec739-239a-4b5a-89e4-2bb89ddbb1df None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:07.799854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec739-239a-4b5a-89e4-2bb89ddbb1df None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:07.800125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec739-239a-4b5a-89e4-2bb89ddbb1df None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:07.800991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec739-239a-4b5a-89e4-2bb89ddbb1df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:07.801296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec739-239a-4b5a-89e4-2bb89ddbb1df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:07.802722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec739-239a-4b5a-89e4-2bb89ddbb1df None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:07.819446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 509/1024] 10.222.0.50 () {56 vars in 1133 bytes} [Fri Apr 29 18:31:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:07.825226 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b56052-3fd5-4e2c-a482-a648fb4ac9ce None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:07.922591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b56052-3fd5-4e2c-a482-a648fb4ac9ce None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:07.956442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57efac-a2e8-4488-936a-3679a43e27fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:07.957194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57efac-a2e8-4488-936a-3679a43e27fe None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:07.957473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57efac-a2e8-4488-936a-3679a43e27fe None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:07.957741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57efac-a2e8-4488-936a-3679a43e27fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:08.027633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa57efac-a2e8-4488-936a-3679a43e27fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:08.029036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa57efac-a2e8-4488-936a-3679a43e27fe None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:08.128537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 516/1025] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:08.134219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec739-239a-4b5a-89e4-2bb89ddbb1df None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:08.237091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec739-239a-4b5a-89e4-2bb89ddbb1df None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:08.271771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2282d-8589-4d78-bc37-099e999598ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:08.272863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2282d-8589-4d78-bc37-099e999598ad None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:08.273457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2282d-8589-4d78-bc37-099e999598ad None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:08.273887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2282d-8589-4d78-bc37-099e999598ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:08.284654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2282d-8589-4d78-bc37-099e999598ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:08.286280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2282d-8589-4d78-bc37-099e999598ad None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-99c2282d-8589-4d78-bc37-099e999598ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:31:08.303925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:08.305205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:08.306113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:31:08.306113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:31:08.306113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:08.306113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:08.306113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 510/1026] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:08.311601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57efac-a2e8-4488-936a-3679a43e27fe None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:08.425292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57efac-a2e8-4488-936a-3679a43e27fe None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:08.459898 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f77cf-99eb-48b5-a65c-8eca43655060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:08.460962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f77cf-99eb-48b5-a65c-8eca43655060 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:08.461421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f77cf-99eb-48b5-a65c-8eca43655060 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:08.461865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f77cf-99eb-48b5-a65c-8eca43655060 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:08.462756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f77cf-99eb-48b5-a65c-8eca43655060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:08.463253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f77cf-99eb-48b5-a65c-8eca43655060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:08.464836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f77cf-99eb-48b5-a65c-8eca43655060 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:08.477571 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 517/1027] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:08.482881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2282d-8589-4d78-bc37-099e999598ad None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:08.579573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2282d-8589-4d78-bc37-099e999598ad None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:08.613330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bc661-f65d-4244-a221-0ee4eab3d227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:08.614127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bc661-f65d-4244-a221-0ee4eab3d227 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:08.614409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bc661-f65d-4244-a221-0ee4eab3d227 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:08.614673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bc661-f65d-4244-a221-0ee4eab3d227 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:08.622672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bc661-f65d-4244-a221-0ee4eab3d227 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:08.623373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bc661-f65d-4244-a221-0ee4eab3d227 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-231bc661-f65d-4244-a221-0ee4eab3d227 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:31:08.636906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:08.637841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:08.638758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:31:08.638758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:31:08.638758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:31:08.638758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:31:08.638758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:31:08.638758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:31:08.638758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:31:08.638758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:08.640865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 511/1028] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:31:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:08.643907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f77cf-99eb-48b5-a65c-8eca43655060 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:08.749198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f77cf-99eb-48b5-a65c-8eca43655060 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:08.783277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbeee1-3323-4475-884a-2d86b19c8024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:08.783943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbeee1-3323-4475-884a-2d86b19c8024 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:08.784220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbeee1-3323-4475-884a-2d86b19c8024 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:08.784518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbeee1-3323-4475-884a-2d86b19c8024 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:08.785317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbeee1-3323-4475-884a-2d86b19c8024 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:08.785626 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbeee1-3323-4475-884a-2d86b19c8024 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:08.787053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbeee1-3323-4475-884a-2d86b19c8024 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:08.799910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 518/1029] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:08.806089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bc661-f65d-4244-a221-0ee4eab3d227 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:08.860452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da3d5c-1c88-42b3-b5de-0e5e27916941 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:08.860676 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da3d5c-1c88-42b3-b5de-0e5e27916941 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:08.860951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da3d5c-1c88-42b3-b5de-0e5e27916941 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:08.862481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 519/1030] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:08.867475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7d016-5a34-4f0a-925b-a6588935dca0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:08.867857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7d016-5a34-4f0a-925b-a6588935dca0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:08.868319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7d016-5a34-4f0a-925b-a6588935dca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:08.885447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-58a7d016-5a34-4f0a-925b-a6588935dca0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:08.886651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-58a7d016-5a34-4f0a-925b-a6588935dca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:08.906858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bc661-f65d-4244-a221-0ee4eab3d227 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:08.943202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6541250-c663-4b4a-a67a-0c98cc8d35dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:08.943892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6541250-c663-4b4a-a67a-0c98cc8d35dc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:08.944175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6541250-c663-4b4a-a67a-0c98cc8d35dc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:08.944447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6541250-c663-4b4a-a67a-0c98cc8d35dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:08.945528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6541250-c663-4b4a-a67a-0c98cc8d35dc None admin] RBAC: Adding query filter params (role.id=6f04dd8fb0cf4f47a384d0fcca2217ea, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=341358254d884b278d7fa19d95c2dc2d) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:08.945846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6541250-c663-4b4a-a67a-0c98cc8d35dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:08.947337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6541250-c663-4b4a-a67a-0c98cc8d35dc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:08.964553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 512/1031] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:31:08 2022] GET /identity/v3/role_assignments?user.id=341358254d884b278d7fa19d95c2dc2d&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=6f04dd8fb0cf4f47a384d0fcca2217ea => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:08.964794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 520/1032] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:08.981788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6541250-c663-4b4a-a67a-0c98cc8d35dc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:09.097462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6541250-c663-4b4a-a67a-0c98cc8d35dc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:09.136178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef62d8a-964b-4678-96d2-817427361078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:09.136980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef62d8a-964b-4678-96d2-817427361078 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:09.137238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef62d8a-964b-4678-96d2-817427361078 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:09.137511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef62d8a-964b-4678-96d2-817427361078 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:09.138736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef62d8a-964b-4678-96d2-817427361078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:09.140201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef62d8a-964b-4678-96d2-817427361078 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:09.141962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 513/1033] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:09.142865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2830a3-d233-4790-948c-250261c2b055 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:09.143154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2830a3-d233-4790-948c-250261c2b055 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:09.143423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2830a3-d233-4790-948c-250261c2b055 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:09.144997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 521/1034] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:31:09.147552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef62d8a-964b-4678-96d2-817427361078 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:09.150557 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae4114-baf0-4d48-89ba-483e49fc2a3f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:09.150844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae4114-baf0-4d48-89ba-483e49fc2a3f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:09.151122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae4114-baf0-4d48-89ba-483e49fc2a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:09.167925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-26ae4114-baf0-4d48-89ba-483e49fc2a3f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:09.169132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26ae4114-baf0-4d48-89ba-483e49fc2a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:09.236017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 522/1035] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:31:09.244577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef62d8a-964b-4678-96d2-817427361078 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:09.259365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae4114-baf0-4d48-89ba-483e49fc2a3f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:09.277727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcd5b-b11b-4bb6-8f9a-22b7b1b92d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:09.278387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcd5b-b11b-4bb6-8f9a-22b7b1b92d98 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:09.278667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcd5b-b11b-4bb6-8f9a-22b7b1b92d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:09.278935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcd5b-b11b-4bb6-8f9a-22b7b1b92d98 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:09.279776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bcd5b-b11b-4bb6-8f9a-22b7b1b92d98 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:09.281101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bcd5b-b11b-4bb6-8f9a-22b7b1b92d98 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:09.316517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 514/1036] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:31:09 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:31:09.358323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae4114-baf0-4d48-89ba-483e49fc2a3f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:09.394827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220650b1-69a2-4ebb-bdbb-d0b3fcf36313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:09.395560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220650b1-69a2-4ebb-bdbb-d0b3fcf36313 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:09.395837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220650b1-69a2-4ebb-bdbb-d0b3fcf36313 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:09.396105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220650b1-69a2-4ebb-bdbb-d0b3fcf36313 None admin] PATH_INFO: `/v3/services/image` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:09.396804 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220650b1-69a2-4ebb-bdbb-d0b3fcf36313 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:09.398231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220650b1-69a2-4ebb-bdbb-d0b3fcf36313 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-220650b1-69a2-4ebb-bdbb-d0b3fcf36313 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:09.410501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:09.411705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Apr 29 18:31:09.412599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:09.416946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 523/1037] 10.222.0.50 () {56 vars in 1105 bytes} [Fri Apr 29 18:31:09 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:31:09.418505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcd5b-b11b-4bb6-8f9a-22b7b1b92d98 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:09.540686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcd5b-b11b-4bb6-8f9a-22b7b1b92d98 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:09.574414 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed01388-c5e3-4895-8e59-8f550a57fd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:09.575091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01388-c5e3-4895-8e59-8f550a57fd2a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:09.575369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01388-c5e3-4895-8e59-8f550a57fd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:09.575636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed01388-c5e3-4895-8e59-8f550a57fd2a None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:09.576340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed01388-c5e3-4895-8e59-8f550a57fd2a None admin] RBAC: Adding query filter params (name=image) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:09.576734 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed01388-c5e3-4895-8e59-8f550a57fd2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:09.578111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed01388-c5e3-4895-8e59-8f550a57fd2a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:09.588045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 515/1038] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:09.589608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220650b1-69a2-4ebb-bdbb-d0b3fcf36313 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:09.694096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220650b1-69a2-4ebb-bdbb-d0b3fcf36313 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:09.729678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d450de-eb3c-4b77-90db-e1dde75ed8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:09.730349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d450de-eb3c-4b77-90db-e1dde75ed8fa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:09.730630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d450de-eb3c-4b77-90db-e1dde75ed8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:09.730901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d450de-eb3c-4b77-90db-e1dde75ed8fa None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:09.731604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d450de-eb3c-4b77-90db-e1dde75ed8fa None admin] RBAC: Adding query filter params (type=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:09.731904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d450de-eb3c-4b77-90db-e1dde75ed8fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:09.733299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d450de-eb3c-4b77-90db-e1dde75ed8fa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:09.738483 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 524/1039] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:09.744928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed01388-c5e3-4895-8e59-8f550a57fd2a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:09.853132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed01388-c5e3-4895-8e59-8f550a57fd2a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:09.887196 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c70ffe-5232-4d51-8777-c6db242ccbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:09.887886 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70ffe-5232-4d51-8777-c6db242ccbfb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:09.888164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70ffe-5232-4d51-8777-c6db242ccbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:09.888435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70ffe-5232-4d51-8777-c6db242ccbfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:09.889257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c70ffe-5232-4d51-8777-c6db242ccbfb None admin] RBAC: Adding query filter params (interface=public, service_id=57f8443e0c1945948632e066184bd5f0, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:09.889558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c70ffe-5232-4d51-8777-c6db242ccbfb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:09.890928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c70ffe-5232-4d51-8777-c6db242ccbfb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:09.896022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 516/1040] 10.222.0.50 () {56 vars in 1251 bytes} [Fri Apr 29 18:31:09 2022] GET /identity/v3/endpoints?service_id=57f8443e0c1945948632e066184bd5f0&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:09.901826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d450de-eb3c-4b77-90db-e1dde75ed8fa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:09.999860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d450de-eb3c-4b77-90db-e1dde75ed8fa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:10.033632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28aa76-3a43-40b5-bcba-b4fa73f756b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:10.034299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28aa76-3a43-40b5-bcba-b4fa73f756b2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:10.034577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28aa76-3a43-40b5-bcba-b4fa73f756b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:10.034851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28aa76-3a43-40b5-bcba-b4fa73f756b2 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:10.035539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28aa76-3a43-40b5-bcba-b4fa73f756b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:10.036966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28aa76-3a43-40b5-bcba-b4fa73f756b2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:10.043785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 525/1041] 10.222.0.50 () {56 vars in 1087 bytes} [Fri Apr 29 18:31:09 2022] GET /identity/v3/services => generated 1957 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:31:11.224575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11ded7-d9a6-4f3f-9133-016106661f95 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:11.225169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11ded7-d9a6-4f3f-9133-016106661f95 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:11.225169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11ded7-d9a6-4f3f-9133-016106661f95 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:11.226562 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 517/1042] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:11.232261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e873b-c2a1-471f-9cef-525c56f7be6d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:11.232728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e873b-c2a1-471f-9cef-525c56f7be6d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:11.232831 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e873b-c2a1-471f-9cef-525c56f7be6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:11.250252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4c3e873b-c2a1-471f-9cef-525c56f7be6d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:11.251435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4c3e873b-c2a1-471f-9cef-525c56f7be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:11.324565 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 526/1043] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:11.342812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11ded7-d9a6-4f3f-9133-016106661f95 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:11.448296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11ded7-d9a6-4f3f-9133-016106661f95 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:11.483645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a73b1-5d46-492a-8031-7aabecd9042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:11.484383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a73b1-5d46-492a-8031-7aabecd9042c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:11.484689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a73b1-5d46-492a-8031-7aabecd9042c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:11.484956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a73b1-5d46-492a-8031-7aabecd9042c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:11.494742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a73b1-5d46-492a-8031-7aabecd9042c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f05a73b1-5d46-492a-8031-7aabecd9042c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:31:11.502701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:11.503637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:31:11.504514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:11.508959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 518/1044] 10.222.0.50 () {56 vars in 1120 bytes} [Fri Apr 29 18:31:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:11.510173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e873b-c2a1-471f-9cef-525c56f7be6d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:11.605218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95202-ee35-479f-879c-840e2baa4912 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:11.605614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95202-ee35-479f-879c-840e2baa4912 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:11.605744 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95202-ee35-479f-879c-840e2baa4912 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:11.607263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 519/1045] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:31:11.608429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e873b-c2a1-471f-9cef-525c56f7be6d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:11.611968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89136096-32a8-49be-abf2-44f50b59383e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:11.612250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89136096-32a8-49be-abf2-44f50b59383e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:11.612516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89136096-32a8-49be-abf2-44f50b59383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:11.634613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-89136096-32a8-49be-abf2-44f50b59383e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:11.635748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-89136096-32a8-49be-abf2-44f50b59383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:11.643697 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcfc6e-9f0c-4af8-8552-a315624b7b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:11.644364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcfc6e-9f0c-4af8-8552-a315624b7b73 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:11.644666 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcfc6e-9f0c-4af8-8552-a315624b7b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:11.644921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fcfc6e-9f0c-4af8-8552-a315624b7b73 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:11.645672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcfc6e-9f0c-4af8-8552-a315624b7b73 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:11.645967 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcfc6e-9f0c-4af8-8552-a315624b7b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:11.647332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fcfc6e-9f0c-4af8-8552-a315624b7b73 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:11.660526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 527/1046] 10.222.0.50 () {56 vars in 1115 bytes} [Fri Apr 29 18:31:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:11.661927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcfc6e-9f0c-4af8-8552-a315624b7b73 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:11.702669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 520/1047] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:31:11.727438 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89136096-32a8-49be-abf2-44f50b59383e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:11.758907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fcfc6e-9f0c-4af8-8552-a315624b7b73 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:11.791615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270621b-f208-4023-8e0e-2c4d956ee65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:11.792409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270621b-f208-4023-8e0e-2c4d956ee65f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:11.792710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270621b-f208-4023-8e0e-2c4d956ee65f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:11.792974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270621b-f208-4023-8e0e-2c4d956ee65f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:11.803229 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270621b-f208-4023-8e0e-2c4d956ee65f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:11.804918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270621b-f208-4023-8e0e-2c4d956ee65f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5270621b-f208-4023-8e0e-2c4d956ee65f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:31:11.821834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:11.822778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:11.829074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:31:11.829074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:31:11.829074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:11.829074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:11.829074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 528/1048] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:11.829356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89136096-32a8-49be-abf2-44f50b59383e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:11.829432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270621b-f208-4023-8e0e-2c4d956ee65f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:11.859835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8ce3-41ce-4325-9126-011ef542be5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:11.860677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8ce3-41ce-4325-9126-011ef542be5e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:11.860956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8ce3-41ce-4325-9126-011ef542be5e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:11.861222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8ce3-41ce-4325-9126-011ef542be5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:11.862367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a8ce3-41ce-4325-9126-011ef542be5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:11.863792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a8ce3-41ce-4325-9126-011ef542be5e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:11.865665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 521/1049] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:11.871088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8ce3-41ce-4325-9126-011ef542be5e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:11.925019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270621b-f208-4023-8e0e-2c4d956ee65f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:11.957770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6997e-e531-491e-b106-1e1ec45db3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:11.958432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6997e-e531-491e-b106-1e1ec45db3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:11.958710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6997e-e531-491e-b106-1e1ec45db3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:11.958978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6997e-e531-491e-b106-1e1ec45db3e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:11.959690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6997e-e531-491e-b106-1e1ec45db3e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:11.959990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6997e-e531-491e-b106-1e1ec45db3e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:11.961725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6997e-e531-491e-b106-1e1ec45db3e5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:11.965968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8ce3-41ce-4325-9126-011ef542be5e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:11.978921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 529/1050] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:11.979803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6997e-e531-491e-b106-1e1ec45db3e5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:11.999976 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f37ca-5ebf-4d97-a2b1-8d9497110b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:12.000673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f37ca-5ebf-4d97-a2b1-8d9497110b3c None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.000951 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f37ca-5ebf-4d97-a2b1-8d9497110b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.001221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f37ca-5ebf-4d97-a2b1-8d9497110b3c None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.001966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f37ca-5ebf-4d97-a2b1-8d9497110b3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:12.003434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f37ca-5ebf-4d97-a2b1-8d9497110b3c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:12.025897 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 522/1051] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:31:11 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:31:12.083335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6997e-e531-491e-b106-1e1ec45db3e5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:12.123117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dff2c-2d9b-49da-b873-995cfa30f37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:12.124267 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dff2c-2d9b-49da-b873-995cfa30f37d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.124755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dff2c-2d9b-49da-b873-995cfa30f37d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.125247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dff2c-2d9b-49da-b873-995cfa30f37d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.134560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979dff2c-2d9b-49da-b873-995cfa30f37d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:12.136437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979dff2c-2d9b-49da-b873-995cfa30f37d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-979dff2c-2d9b-49da-b873-995cfa30f37d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:31:12.153591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:12.154689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:12.155646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:12.155646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:12.155646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:31:12.155646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:31:12.155646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:31:12.155646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:31:12.155646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:31:12.155646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:31:12.155646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:31:12.155646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:12.156482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 530/1052] 10.222.0.50 () {56 vars in 1093 bytes} [Fri Apr 29 18:31:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:31:12.161711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f37ca-5ebf-4d97-a2b1-8d9497110b3c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:12.266426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f37ca-5ebf-4d97-a2b1-8d9497110b3c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:12.300534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed82e6-cb55-4991-92b0-184e245c825e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:12.301170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed82e6-cb55-4991-92b0-184e245c825e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.301445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed82e6-cb55-4991-92b0-184e245c825e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.301708 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed82e6-cb55-4991-92b0-184e245c825e None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.302540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed82e6-cb55-4991-92b0-184e245c825e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:12.302842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed82e6-cb55-4991-92b0-184e245c825e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:12.304364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed82e6-cb55-4991-92b0-184e245c825e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:12.322119 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 523/1053] 10.222.0.50 () {56 vars in 1133 bytes} [Fri Apr 29 18:31:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:12.326166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371005-7b02-4a20-865b-dfe171982173 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.326619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371005-7b02-4a20-865b-dfe171982173 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.326974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371005-7b02-4a20-865b-dfe171982173 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.327565 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed82e6-cb55-4991-92b0-184e245c825e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:12.328373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 531/1054] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:12.333443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614811fa-8327-4d47-b3f1-315465063336 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.333725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614811fa-8327-4d47-b3f1-315465063336 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.334002 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614811fa-8327-4d47-b3f1-315465063336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.351047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-614811fa-8327-4d47-b3f1-315465063336 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:12.352349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-614811fa-8327-4d47-b3f1-315465063336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:12.422163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 532/1055] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:31:12.427344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed82e6-cb55-4991-92b0-184e245c825e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:12.444333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614811fa-8327-4d47-b3f1-315465063336 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:12.459976 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396718f1-9a12-4126-8896-5107db5789b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:12.461051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396718f1-9a12-4126-8896-5107db5789b4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.461326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396718f1-9a12-4126-8896-5107db5789b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.461584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396718f1-9a12-4126-8896-5107db5789b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.475773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396718f1-9a12-4126-8896-5107db5789b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:12.477609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396718f1-9a12-4126-8896-5107db5789b4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-396718f1-9a12-4126-8896-5107db5789b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:31:12.497192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:12.498145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:12.499146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:31:12.499146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:31:12.499146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:12.499146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:12.499146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 524/1056] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:12.503999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396718f1-9a12-4126-8896-5107db5789b4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:12.541752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614811fa-8327-4d47-b3f1-315465063336 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:12.573958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44e9bb-c0cd-4722-89c3-15c016d904f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:12.574676 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44e9bb-c0cd-4722-89c3-15c016d904f9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.574953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44e9bb-c0cd-4722-89c3-15c016d904f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.575220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44e9bb-c0cd-4722-89c3-15c016d904f9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.575864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44e9bb-c0cd-4722-89c3-15c016d904f9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:12.577245 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44e9bb-c0cd-4722-89c3-15c016d904f9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3d44e9bb-c0cd-4722-89c3-15c016d904f9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:12.588611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:12.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:12.590524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 533/1057] 10.222.0.50 () {56 vars in 1105 bytes} [Fri Apr 29 18:31:12 2022] GET /identity/v3/services/image => generated 86 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:31:12.596264 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44e9bb-c0cd-4722-89c3-15c016d904f9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:12.611482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396718f1-9a12-4126-8896-5107db5789b4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:12.643851 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357411d-87b9-4985-a20d-1aeca6197a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:12.644543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357411d-87b9-4985-a20d-1aeca6197a47 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.644826 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357411d-87b9-4985-a20d-1aeca6197a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.645095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357411d-87b9-4985-a20d-1aeca6197a47 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.645802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357411d-87b9-4985-a20d-1aeca6197a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:12.646101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357411d-87b9-4985-a20d-1aeca6197a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:12.647451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357411d-87b9-4985-a20d-1aeca6197a47 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:12.658366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 525/1058] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:12.663981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357411d-87b9-4985-a20d-1aeca6197a47 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:12.698977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44e9bb-c0cd-4722-89c3-15c016d904f9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:12.731122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f202fe-08cc-4d43-b6b3-48d6132e9df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:12.731781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f202fe-08cc-4d43-b6b3-48d6132e9df8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.732057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f202fe-08cc-4d43-b6b3-48d6132e9df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.732325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f202fe-08cc-4d43-b6b3-48d6132e9df8 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.733115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f202fe-08cc-4d43-b6b3-48d6132e9df8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:12.733417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f202fe-08cc-4d43-b6b3-48d6132e9df8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:12.734824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f202fe-08cc-4d43-b6b3-48d6132e9df8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:12.744781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 534/1059] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:12.745672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f202fe-08cc-4d43-b6b3-48d6132e9df8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:12.756988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357411d-87b9-4985-a20d-1aeca6197a47 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:12.791174 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b39313-728b-4088-ae43-639d2d339e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:12.791411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b39313-728b-4088-ae43-639d2d339e7e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.791576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b39313-728b-4088-ae43-639d2d339e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.791939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b39313-728b-4088-ae43-639d2d339e7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.798977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b39313-728b-4088-ae43-639d2d339e7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:12.800580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b39313-728b-4088-ae43-639d2d339e7e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-75b39313-728b-4088-ae43-639d2d339e7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:31:12.816302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:12.817299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:12.823684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:31:12.823684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:31:12.823684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:31:12.823684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:31:12.823684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:31:12.823684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:31:12.823684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:31:12.823684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:12.823684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 526/1060] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:31:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:31:12.823684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b39313-728b-4088-ae43-639d2d339e7e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:12.857398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f202fe-08cc-4d43-b6b3-48d6132e9df8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:12.893313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fcd3c-edc8-4830-bbd8-cd0269535422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:12.893980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fcd3c-edc8-4830-bbd8-cd0269535422 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.894265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fcd3c-edc8-4830-bbd8-cd0269535422 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.894542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fcd3c-edc8-4830-bbd8-cd0269535422 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.895250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fcd3c-edc8-4830-bbd8-cd0269535422 None admin] RBAC: Adding query filter params (type=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:12.895558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fcd3c-edc8-4830-bbd8-cd0269535422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:12.897131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fcd3c-edc8-4830-bbd8-cd0269535422 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:12.904538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 535/1061] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:12.909636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fcd3c-edc8-4830-bbd8-cd0269535422 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:12.946609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b39313-728b-4088-ae43-639d2d339e7e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:12.990407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c688f-4066-4021-b5b7-ec0c6f35f77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:12.991097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c688f-4066-4021-b5b7-ec0c6f35f77d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:12.991538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c688f-4066-4021-b5b7-ec0c6f35f77d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:12.991892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c688f-4066-4021-b5b7-ec0c6f35f77d None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:12.993108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c688f-4066-4021-b5b7-ec0c6f35f77d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:12.993733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c688f-4066-4021-b5b7-ec0c6f35f77d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:12.995688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c688f-4066-4021-b5b7-ec0c6f35f77d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:13.013294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 527/1062] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:31:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:13.014200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c688f-4066-4021-b5b7-ec0c6f35f77d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:13.034621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fcd3c-edc8-4830-bbd8-cd0269535422 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:13.068267 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f74be-1ad3-42d4-8007-01b25af4fece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:13.069040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f74be-1ad3-42d4-8007-01b25af4fece None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:13.069390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f74be-1ad3-42d4-8007-01b25af4fece None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:13.069783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f74be-1ad3-42d4-8007-01b25af4fece None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:13.070573 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f74be-1ad3-42d4-8007-01b25af4fece None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:13.072013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f74be-1ad3-42d4-8007-01b25af4fece None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:13.094235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 536/1063] 10.222.0.50 () {60 vars in 1144 bytes} [Fri Apr 29 18:31:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:31:13.131211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c688f-4066-4021-b5b7-ec0c6f35f77d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:13.178167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c002-5cfa-48b2-81da-c53bb8e30b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:13.179581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c002-5cfa-48b2-81da-c53bb8e30b5e None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:13.180175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c002-5cfa-48b2-81da-c53bb8e30b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:13.180532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c002-5cfa-48b2-81da-c53bb8e30b5e None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/341358254d884b278d7fa19d95c2dc2d/roles/6f04dd8fb0cf4f47a384d0fcca2217ea` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:13.183055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2c002-5cfa-48b2-81da-c53bb8e30b5e None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=341358254d884b278d7fa19d95c2dc2d, role_id=6f04dd8fb0cf4f47a384d0fcca2217ea)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:13.184691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2c002-5cfa-48b2-81da-c53bb8e30b5e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:13.204786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 528/1064] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:31:13 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/341358254d884b278d7fa19d95c2dc2d/roles/6f04dd8fb0cf4f47a384d0fcca2217ea => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:31:14.416455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272bf1-c693-4cbf-866b-0ebdc942ac81 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:14.417008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272bf1-c693-4cbf-866b-0ebdc942ac81 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:14.417116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272bf1-c693-4cbf-866b-0ebdc942ac81 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:14.418493 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 537/1065] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:14.423919 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7eb485-2afd-4e5f-a874-f4276fdf1b13 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:14.424357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7eb485-2afd-4e5f-a874-f4276fdf1b13 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:14.424451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7eb485-2afd-4e5f-a874-f4276fdf1b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:14.441609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dd7eb485-2afd-4e5f-a874-f4276fdf1b13 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:14.442773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dd7eb485-2afd-4e5f-a874-f4276fdf1b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:14.510199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 529/1066] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:14.534439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272bf1-c693-4cbf-866b-0ebdc942ac81 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:14.644609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272bf1-c693-4cbf-866b-0ebdc942ac81 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:14.678690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f3b4e-7042-4f48-8987-b0073b68653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:14.679765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f3b4e-7042-4f48-8987-b0073b68653d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:14.679900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f3b4e-7042-4f48-8987-b0073b68653d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:14.680156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f3b4e-7042-4f48-8987-b0073b68653d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:14.690670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f3b4e-7042-4f48-8987-b0073b68653d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a99f3b4e-7042-4f48-8987-b0073b68653d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:31:14.698346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:14.699291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:14.700624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 538/1067] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:31:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:14.705761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7eb485-2afd-4e5f-a874-f4276fdf1b13 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:14.803511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7eb485-2afd-4e5f-a874-f4276fdf1b13 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:14.838663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0edf53-a14d-407a-b9d0-1e3f751319cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:14.839335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0edf53-a14d-407a-b9d0-1e3f751319cc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:14.839615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0edf53-a14d-407a-b9d0-1e3f751319cc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:14.839882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0edf53-a14d-407a-b9d0-1e3f751319cc None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:14.840650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0edf53-a14d-407a-b9d0-1e3f751319cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:14.840951 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0edf53-a14d-407a-b9d0-1e3f751319cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:14.842686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0edf53-a14d-407a-b9d0-1e3f751319cc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:14.851241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 530/1068] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:31:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:14.856984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f3b4e-7042-4f48-8987-b0073b68653d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:14.956933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f3b4e-7042-4f48-8987-b0073b68653d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:14.990331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d739b-7d28-420c-9519-dc32bc14fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:14.991207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d739b-7d28-420c-9519-dc32bc14fe13 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:14.991482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d739b-7d28-420c-9519-dc32bc14fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:14.991902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d739b-7d28-420c-9519-dc32bc14fe13 None admin] PATH_INFO: `/v3/users/d83fccadf4524e039f0ff1cda0089012` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.005587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d739b-7d28-420c-9519-dc32bc14fe13 None admin] RBAC: Authorizing `identity:get_user(user_id=d83fccadf4524e039f0ff1cda0089012)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:15.007027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d739b-7d28-420c-9519-dc32bc14fe13 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:15.008992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 539/1069] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:14 2022] GET /identity/v3/users/d83fccadf4524e039f0ff1cda0089012 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:15.014803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0edf53-a14d-407a-b9d0-1e3f751319cc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:15.115072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0edf53-a14d-407a-b9d0-1e3f751319cc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:15.149113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ebc5f-7960-48c6-89b0-9548424050b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:15.149766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ebc5f-7960-48c6-89b0-9548424050b8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:15.150046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ebc5f-7960-48c6-89b0-9548424050b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:15.150327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ebc5f-7960-48c6-89b0-9548424050b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.219704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ebc5f-7960-48c6-89b0-9548424050b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:15.221137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ebc5f-7960-48c6-89b0-9548424050b8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:15.325446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 531/1070] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:15.332329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d739b-7d28-420c-9519-dc32bc14fe13 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:15.409280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b85d27-1b65-4381-8328-fb0016b385de None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:15.409533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b85d27-1b65-4381-8328-fb0016b385de None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:15.409821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b85d27-1b65-4381-8328-fb0016b385de None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.411357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 532/1071] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:31:15.417032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe3315-fa24-4c4d-a730-1e5a48cb6dfe None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:15.417322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe3315-fa24-4c4d-a730-1e5a48cb6dfe None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:15.417626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe3315-fa24-4c4d-a730-1e5a48cb6dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.434852 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-2cfe3315-fa24-4c4d-a730-1e5a48cb6dfe None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:15.436823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2cfe3315-fa24-4c4d-a730-1e5a48cb6dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:15.436969 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d739b-7d28-420c-9519-dc32bc14fe13 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:15.471609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750d48a-7729-4b5d-a1e7-f73f8b61584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:15.472388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750d48a-7729-4b5d-a1e7-f73f8b61584d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:15.472664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750d48a-7729-4b5d-a1e7-f73f8b61584d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:15.472960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750d48a-7729-4b5d-a1e7-f73f8b61584d None admin] PATH_INFO: `/v3/projects/58bbd3bde9d743288dd7804c0c350525` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.474199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750d48a-7729-4b5d-a1e7-f73f8b61584d None admin] RBAC: Authorizing `identity:get_project(project_id=58bbd3bde9d743288dd7804c0c350525)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:15.475629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750d48a-7729-4b5d-a1e7-f73f8b61584d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:15.478076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 540/1072] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:31:15 2022] GET /identity/v3/projects/58bbd3bde9d743288dd7804c0c350525 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:15.483882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750d48a-7729-4b5d-a1e7-f73f8b61584d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:15.505148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 533/1073] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:31:15.506470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115800-647e-4495-991b-c001a767c7d9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:15.506755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115800-647e-4495-991b-c001a767c7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:15.507026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115800-647e-4495-991b-c001a767c7d9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.509079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 534/1074] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:15.513944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a208c-9628-4eb6-a125-c34f75964514 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:15.514231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a208c-9628-4eb6-a125-c34f75964514 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:15.514506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a208c-9628-4eb6-a125-c34f75964514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.531094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4d1a208c-9628-4eb6-a125-c34f75964514 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:15.532271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4d1a208c-9628-4eb6-a125-c34f75964514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:15.580404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750d48a-7729-4b5d-a1e7-f73f8b61584d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:15.594873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 535/1075] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:31:15.595689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a208c-9628-4eb6-a125-c34f75964514 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:15.616083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0134929-7b44-4358-8200-49b6a7852ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:15.617318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0134929-7b44-4358-8200-49b6a7852ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:15.617646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0134929-7b44-4358-8200-49b6a7852ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:15.618202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0134929-7b44-4358-8200-49b6a7852ee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.618899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0134929-7b44-4358-8200-49b6a7852ee1 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=58bbd3bde9d743288dd7804c0c350525, user.id=d83fccadf4524e039f0ff1cda0089012) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:15.619218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0134929-7b44-4358-8200-49b6a7852ee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:15.620786 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0134929-7b44-4358-8200-49b6a7852ee1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:15.633102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 541/1076] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:31:15 2022] GET /identity/v3/role_assignments?user.id=d83fccadf4524e039f0ff1cda0089012&scope.project.id=58bbd3bde9d743288dd7804c0c350525&role.id=e55ef11844a742f5889243f189bd59bc => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:15.633102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0134929-7b44-4358-8200-49b6a7852ee1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:15.707208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a208c-9628-4eb6-a125-c34f75964514 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:15.742542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0134929-7b44-4358-8200-49b6a7852ee1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:15.745514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c24693-4b45-432a-a837-289447cb5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:15.746196 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c24693-4b45-432a-a837-289447cb5306 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:15.746475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c24693-4b45-432a-a837-289447cb5306 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:15.746744 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c24693-4b45-432a-a837-289447cb5306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.781164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fd4a7-4c44-4ef3-8b43-8c6f2364ed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:15.781998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fd4a7-4c44-4ef3-8b43-8c6f2364ed1f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:15.782276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fd4a7-4c44-4ef3-8b43-8c6f2364ed1f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:15.782546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fd4a7-4c44-4ef3-8b43-8c6f2364ed1f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.794251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fd4a7-4c44-4ef3-8b43-8c6f2364ed1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-024fd4a7-4c44-4ef3-8b43-8c6f2364ed1f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:31:15.802668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:15.803803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:15.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 542/1077] 10.222.0.50 () {56 vars in 1120 bytes} [Fri Apr 29 18:31:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:15.810883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fd4a7-4c44-4ef3-8b43-8c6f2364ed1f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:15.820154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c24693-4b45-432a-a837-289447cb5306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:15.821661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c24693-4b45-432a-a837-289447cb5306 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:15.910256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fd4a7-4c44-4ef3-8b43-8c6f2364ed1f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:15.919007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 536/1078] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:15.924823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c24693-4b45-432a-a837-289447cb5306 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:15.947175 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599108c-a52e-4728-8040-d639f11c8c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:15.947858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599108c-a52e-4728-8040-d639f11c8c3c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:15.948136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599108c-a52e-4728-8040-d639f11c8c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:15.948484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599108c-a52e-4728-8040-d639f11c8c3c None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:15.949195 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599108c-a52e-4728-8040-d639f11c8c3c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:15.949483 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599108c-a52e-4728-8040-d639f11c8c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:15.950849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599108c-a52e-4728-8040-d639f11c8c3c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:15.959979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 543/1079] 10.222.0.50 () {56 vars in 1115 bytes} [Fri Apr 29 18:31:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:15.965986 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599108c-a52e-4728-8040-d639f11c8c3c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:16.028477 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c24693-4b45-432a-a837-289447cb5306 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:16.062721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada7b03-8501-4585-9a9a-3945552b7c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:16.063523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada7b03-8501-4585-9a9a-3945552b7c84 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:16.063798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada7b03-8501-4585-9a9a-3945552b7c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:16.064070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada7b03-8501-4585-9a9a-3945552b7c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:16.068954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599108c-a52e-4728-8040-d639f11c8c3c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:16.074813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada7b03-8501-4585-9a9a-3945552b7c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:16.076231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada7b03-8501-4585-9a9a-3945552b7c84 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-9ada7b03-8501-4585-9a9a-3945552b7c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:31:16.093957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:16.094898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:16.095797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:31:16.095797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:31:16.095797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:16.095797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:16.095797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 537/1080] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:31:16.101336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada7b03-8501-4585-9a9a-3945552b7c84 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:16.104262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a706ba-1448-4ee9-95cf-ff2752947af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:16.105097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a706ba-1448-4ee9-95cf-ff2752947af4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:16.105370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a706ba-1448-4ee9-95cf-ff2752947af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:16.105637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a706ba-1448-4ee9-95cf-ff2752947af4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:16.116079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a706ba-1448-4ee9-95cf-ff2752947af4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:16.117762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a706ba-1448-4ee9-95cf-ff2752947af4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c8a706ba-1448-4ee9-95cf-ff2752947af4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:31:16.134409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:16.135355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:16.136285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:31:16.136285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:31:16.136285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:16.136285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:16.136285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 544/1081] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:31:16.141204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a706ba-1448-4ee9-95cf-ff2752947af4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:16.200186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada7b03-8501-4585-9a9a-3945552b7c84 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:16.233861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85734f5-8d98-4a05-a50b-87d18e59a581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:16.234532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85734f5-8d98-4a05-a50b-87d18e59a581 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:16.234813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85734f5-8d98-4a05-a50b-87d18e59a581 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:16.235093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85734f5-8d98-4a05-a50b-87d18e59a581 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:16.235792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85734f5-8d98-4a05-a50b-87d18e59a581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:16.236094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85734f5-8d98-4a05-a50b-87d18e59a581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:16.237193 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a706ba-1448-4ee9-95cf-ff2752947af4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:16.237491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85734f5-8d98-4a05-a50b-87d18e59a581 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:16.260531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 538/1082] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:16.288663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1077a9-82cc-4265-bdd2-79ba45d95aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:16.289478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1077a9-82cc-4265-bdd2-79ba45d95aad None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:16.289618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1077a9-82cc-4265-bdd2-79ba45d95aad None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:16.289968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1077a9-82cc-4265-bdd2-79ba45d95aad None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:16.290890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1077a9-82cc-4265-bdd2-79ba45d95aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:16.291253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1077a9-82cc-4265-bdd2-79ba45d95aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:16.292802 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1077a9-82cc-4265-bdd2-79ba45d95aad None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:16.307330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 545/1083] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:16.313308 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85734f5-8d98-4a05-a50b-87d18e59a581 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:16.425462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85734f5-8d98-4a05-a50b-87d18e59a581 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:16.462308 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d55f-15dc-42fc-a3ad-eb9f39da1cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:16.463138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d55f-15dc-42fc-a3ad-eb9f39da1cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:16.463392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d55f-15dc-42fc-a3ad-eb9f39da1cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:16.463660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d55f-15dc-42fc-a3ad-eb9f39da1cb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:16.472319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6d55f-15dc-42fc-a3ad-eb9f39da1cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:16.473801 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6d55f-15dc-42fc-a3ad-eb9f39da1cb0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-33c6d55f-15dc-42fc-a3ad-eb9f39da1cb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:31:16.488019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:16.489264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:16.490396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 539/1084] 10.222.0.50 () {56 vars in 1093 bytes} [Fri Apr 29 18:31:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:16.496019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1077a9-82cc-4265-bdd2-79ba45d95aad None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:16.595150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1077a9-82cc-4265-bdd2-79ba45d95aad None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:16.627864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ba8c6-d9e0-455a-9a82-0b2eac5d28d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:16.628586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ba8c6-d9e0-455a-9a82-0b2eac5d28d0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:16.628866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ba8c6-d9e0-455a-9a82-0b2eac5d28d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:16.629132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ba8c6-d9e0-455a-9a82-0b2eac5d28d0 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:16.629968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ba8c6-d9e0-455a-9a82-0b2eac5d28d0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:16.630273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ba8c6-d9e0-455a-9a82-0b2eac5d28d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:16.631706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ba8c6-d9e0-455a-9a82-0b2eac5d28d0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:16.650063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 546/1085] 10.222.0.50 () {56 vars in 1133 bytes} [Fri Apr 29 18:31:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:16.655839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d55f-15dc-42fc-a3ad-eb9f39da1cb0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:16.757412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d55f-15dc-42fc-a3ad-eb9f39da1cb0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:16.791218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b8257-35bf-4852-a791-e905f70b9d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:16.791867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b8257-35bf-4852-a791-e905f70b9d01 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:16.792148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b8257-35bf-4852-a791-e905f70b9d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:16.792406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b8257-35bf-4852-a791-e905f70b9d01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:16.858474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b8257-35bf-4852-a791-e905f70b9d01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:16.859861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b8257-35bf-4852-a791-e905f70b9d01 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:16.954655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 540/1086] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:16.960766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ba8c6-d9e0-455a-9a82-0b2eac5d28d0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:17.058802 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ba8c6-d9e0-455a-9a82-0b2eac5d28d0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:17.096089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef40f4-cbda-4009-9a0d-3ab4c4d9ca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:17.096913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef40f4-cbda-4009-9a0d-3ab4c4d9ca27 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:17.097194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef40f4-cbda-4009-9a0d-3ab4c4d9ca27 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:17.097462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef40f4-cbda-4009-9a0d-3ab4c4d9ca27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:17.107691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef40f4-cbda-4009-9a0d-3ab4c4d9ca27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:17.109265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef40f4-cbda-4009-9a0d-3ab4c4d9ca27 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-03ef40f4-cbda-4009-9a0d-3ab4c4d9ca27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:31:17.126252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:17.127209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:17.128136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:31:17.128136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:31:17.128136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:31:17.128136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:17.129012 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 547/1087] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:17.133394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b8257-35bf-4852-a791-e905f70b9d01 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:17.230229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b8257-35bf-4852-a791-e905f70b9d01 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:17.263945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1064c-4725-435f-b43d-a6ba9670a533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:17.264650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1064c-4725-435f-b43d-a6ba9670a533 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:17.264933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1064c-4725-435f-b43d-a6ba9670a533 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:17.265204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1064c-4725-435f-b43d-a6ba9670a533 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:17.265924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1064c-4725-435f-b43d-a6ba9670a533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:17.266227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1064c-4725-435f-b43d-a6ba9670a533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:17.267587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1064c-4725-435f-b43d-a6ba9670a533 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:17.279009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 541/1088] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:17.284748 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef40f4-cbda-4009-9a0d-3ab4c4d9ca27 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:17.389613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef40f4-cbda-4009-9a0d-3ab4c4d9ca27 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:17.423517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772b215-8b17-4273-9efc-021e22542169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:17.424319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772b215-8b17-4273-9efc-021e22542169 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:17.424629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772b215-8b17-4273-9efc-021e22542169 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:17.424901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772b215-8b17-4273-9efc-021e22542169 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:17.432173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772b215-8b17-4273-9efc-021e22542169 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:17.433640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772b215-8b17-4273-9efc-021e22542169 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-2772b215-8b17-4273-9efc-021e22542169 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:31:17.447270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:17.448335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:17.449466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:31:17.449466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:31:17.449466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:31:17.449466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:31:17.449466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:31:17.449466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:31:17.449466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:31:17.449466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:17.452684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 548/1089] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:31:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:17.454331 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1064c-4725-435f-b43d-a6ba9670a533 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:17.552984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1064c-4725-435f-b43d-a6ba9670a533 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:17.586505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc41b6-b5fc-4aec-9510-8b4afc76110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:17.587164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc41b6-b5fc-4aec-9510-8b4afc76110d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:17.587450 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc41b6-b5fc-4aec-9510-8b4afc76110d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:17.587721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc41b6-b5fc-4aec-9510-8b4afc76110d None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:17.588514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc41b6-b5fc-4aec-9510-8b4afc76110d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:17.588924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc41b6-b5fc-4aec-9510-8b4afc76110d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:17.590366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc41b6-b5fc-4aec-9510-8b4afc76110d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:17.602647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 542/1090] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:31:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:17.608865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772b215-8b17-4273-9efc-021e22542169 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:17.710989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772b215-8b17-4273-9efc-021e22542169 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:17.745689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99ada3-f702-4fe9-919c-7cddc6cebfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:17.746334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99ada3-f702-4fe9-919c-7cddc6cebfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:17.746611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99ada3-f702-4fe9-919c-7cddc6cebfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:17.746879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99ada3-f702-4fe9-919c-7cddc6cebfc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:17.747696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99ada3-f702-4fe9-919c-7cddc6cebfc4 None admin] RBAC: Adding query filter params (role.id=6f04dd8fb0cf4f47a384d0fcca2217ea, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=341358254d884b278d7fa19d95c2dc2d) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:17.748002 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99ada3-f702-4fe9-919c-7cddc6cebfc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:17.749451 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99ada3-f702-4fe9-919c-7cddc6cebfc4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:17.755376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 549/1091] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:31:17 2022] GET /identity/v3/role_assignments?user.id=341358254d884b278d7fa19d95c2dc2d&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=6f04dd8fb0cf4f47a384d0fcca2217ea => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:17.848551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed07a7-3a21-4e4f-8ae2-b4ef0a01da1d None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:17.848836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed07a7-3a21-4e4f-8ae2-b4ef0a01da1d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:17.849167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed07a7-3a21-4e4f-8ae2-b4ef0a01da1d None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:17.850564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 543/1092] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:17.856553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f7e9c-c2d6-4cb0-996a-3bc1efb1a37e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:17.856864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f7e9c-c2d6-4cb0-996a-3bc1efb1a37e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:17.857137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f7e9c-c2d6-4cb0-996a-3bc1efb1a37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:17.874774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0c9f7e9c-c2d6-4cb0-996a-3bc1efb1a37e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:17.875973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c9f7e9c-c2d6-4cb0-996a-3bc1efb1a37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:17.949336 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 550/1093] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:17.973126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed07a7-3a21-4e4f-8ae2-b4ef0a01da1d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:18.074409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed07a7-3a21-4e4f-8ae2-b4ef0a01da1d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:18.107677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2eccf1-d085-4970-abb0-773a02cc1b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:18.108472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2eccf1-d085-4970-abb0-773a02cc1b2e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:18.108775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2eccf1-d085-4970-abb0-773a02cc1b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:18.109041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2eccf1-d085-4970-abb0-773a02cc1b2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:18.119366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2eccf1-d085-4970-abb0-773a02cc1b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-3b2eccf1-d085-4970-abb0-773a02cc1b2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:31:18.126823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:18.127805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:18.128704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 544/1094] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:31:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:18.135023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f7e9c-c2d6-4cb0-996a-3bc1efb1a37e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:18.238751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f7e9c-c2d6-4cb0-996a-3bc1efb1a37e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:18.271886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843d528-2aa0-4268-adf5-b289da0ce003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:18.272581 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843d528-2aa0-4268-adf5-b289da0ce003 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:18.273037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843d528-2aa0-4268-adf5-b289da0ce003 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:18.273869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843d528-2aa0-4268-adf5-b289da0ce003 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:18.274619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843d528-2aa0-4268-adf5-b289da0ce003 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:18.274925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843d528-2aa0-4268-adf5-b289da0ce003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:18.276306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843d528-2aa0-4268-adf5-b289da0ce003 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:18.284519 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 551/1095] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:31:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:18.291477 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2eccf1-d085-4970-abb0-773a02cc1b2e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:18.397334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2eccf1-d085-4970-abb0-773a02cc1b2e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:18.430929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fa4a6-f9db-4d1f-9489-5009ac1425db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:18.431828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fa4a6-f9db-4d1f-9489-5009ac1425db None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:18.432109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fa4a6-f9db-4d1f-9489-5009ac1425db None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:18.432380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fa4a6-f9db-4d1f-9489-5009ac1425db None admin] PATH_INFO: `/v3/users/d83fccadf4524e039f0ff1cda0089012` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:18.433626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fa4a6-f9db-4d1f-9489-5009ac1425db None admin] RBAC: Authorizing `identity:get_user(user_id=d83fccadf4524e039f0ff1cda0089012)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:18.435066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fa4a6-f9db-4d1f-9489-5009ac1425db None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:18.436883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 545/1096] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:18 2022] GET /identity/v3/users/d83fccadf4524e039f0ff1cda0089012 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:18.443255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843d528-2aa0-4268-adf5-b289da0ce003 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:18.543285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843d528-2aa0-4268-adf5-b289da0ce003 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:18.577505 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0c8bf-2143-40be-ac9e-66de16e0466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:18.578379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0c8bf-2143-40be-ac9e-66de16e0466d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:18.578653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0c8bf-2143-40be-ac9e-66de16e0466d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:18.578925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0c8bf-2143-40be-ac9e-66de16e0466d None admin] PATH_INFO: `/v3/projects/58bbd3bde9d743288dd7804c0c350525` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:18.580227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0c8bf-2143-40be-ac9e-66de16e0466d None admin] RBAC: Authorizing `identity:get_project(project_id=58bbd3bde9d743288dd7804c0c350525)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:18.581775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0c8bf-2143-40be-ac9e-66de16e0466d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:18.583480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 552/1097] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:31:18 2022] GET /identity/v3/projects/58bbd3bde9d743288dd7804c0c350525 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:18.590296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fa4a6-f9db-4d1f-9489-5009ac1425db None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:18.691504 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fa4a6-f9db-4d1f-9489-5009ac1425db None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:18.725042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80d4c3-9aad-4ec3-bc5d-50f53235e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:18.726173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80d4c3-9aad-4ec3-bc5d-50f53235e69d None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:18.726504 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80d4c3-9aad-4ec3-bc5d-50f53235e69d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:18.726722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80d4c3-9aad-4ec3-bc5d-50f53235e69d None admin] PATH_INFO: `/v3/projects/58bbd3bde9d743288dd7804c0c350525/users/d83fccadf4524e039f0ff1cda0089012/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:18.728811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80d4c3-9aad-4ec3-bc5d-50f53235e69d None admin] RBAC: Authorizing `identity:create_grant(project_id=58bbd3bde9d743288dd7804c0c350525, user_id=d83fccadf4524e039f0ff1cda0089012, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:18.730245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80d4c3-9aad-4ec3-bc5d-50f53235e69d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:18.746419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 546/1098] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:31:18 2022] PUT /identity/v3/projects/58bbd3bde9d743288dd7804c0c350525/users/d83fccadf4524e039f0ff1cda0089012/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:31:20.934220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30e75a-e35e-4629-9a34-2ab22878a58e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:20.934766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30e75a-e35e-4629-9a34-2ab22878a58e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:20.934766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30e75a-e35e-4629-9a34-2ab22878a58e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:20.936028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 553/1099] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:20.942438 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c5354-dadf-45d2-a41b-e3a3a7e809fa None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:20.943236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c5354-dadf-45d2-a41b-e3a3a7e809fa None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:20.943450 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c5354-dadf-45d2-a41b-e3a3a7e809fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:20.970382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c95c5354-dadf-45d2-a41b-e3a3a7e809fa None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:20.972228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c95c5354-dadf-45d2-a41b-e3a3a7e809fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:21.043695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 547/1100] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:21.072056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30e75a-e35e-4629-9a34-2ab22878a58e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:21.182434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30e75a-e35e-4629-9a34-2ab22878a58e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:21.218342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0ed6a-cd92-4748-b2f5-c578d8fe5050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:21.219119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0ed6a-cd92-4748-b2f5-c578d8fe5050 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:21.219390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0ed6a-cd92-4748-b2f5-c578d8fe5050 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:21.219657 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0ed6a-cd92-4748-b2f5-c578d8fe5050 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:21.230222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f0ed6a-cd92-4748-b2f5-c578d8fe5050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f2f0ed6a-cd92-4748-b2f5-c578d8fe5050 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:31:21.237599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:21.238586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:21.239706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 554/1101] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:31:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:21.245770 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c5354-dadf-45d2-a41b-e3a3a7e809fa None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:21.342217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c5354-dadf-45d2-a41b-e3a3a7e809fa None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:21.375506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8ab2-0c1c-464f-960d-0bbd0ece975f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:21.376150 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8ab2-0c1c-464f-960d-0bbd0ece975f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:21.376425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8ab2-0c1c-464f-960d-0bbd0ece975f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:21.376726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8ab2-0c1c-464f-960d-0bbd0ece975f None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:21.377453 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b8ab2-0c1c-464f-960d-0bbd0ece975f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:21.377757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b8ab2-0c1c-464f-960d-0bbd0ece975f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:21.379113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b8ab2-0c1c-464f-960d-0bbd0ece975f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:21.387069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 548/1102] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:31:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:21.393168 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0ed6a-cd92-4748-b2f5-c578d8fe5050 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:21.496390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0ed6a-cd92-4748-b2f5-c578d8fe5050 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:21.529532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220aecc-17ca-46ad-88fc-b4c324c8d50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:21.530393 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220aecc-17ca-46ad-88fc-b4c324c8d50e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:21.530665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220aecc-17ca-46ad-88fc-b4c324c8d50e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:21.530937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220aecc-17ca-46ad-88fc-b4c324c8d50e None admin] PATH_INFO: `/v3/users/d83fccadf4524e039f0ff1cda0089012` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:21.532222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220aecc-17ca-46ad-88fc-b4c324c8d50e None admin] RBAC: Authorizing `identity:get_user(user_id=d83fccadf4524e039f0ff1cda0089012)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:21.533684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220aecc-17ca-46ad-88fc-b4c324c8d50e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:21.535293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 555/1103] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:21 2022] GET /identity/v3/users/d83fccadf4524e039f0ff1cda0089012 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:21.542686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8ab2-0c1c-464f-960d-0bbd0ece975f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:21.644659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8ab2-0c1c-464f-960d-0bbd0ece975f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:21.683388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adbc6b-11c8-4703-ad5a-828c9e823489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:21.684033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adbc6b-11c8-4703-ad5a-828c9e823489 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:21.684318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adbc6b-11c8-4703-ad5a-828c9e823489 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:21.684592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adbc6b-11c8-4703-ad5a-828c9e823489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:21.757640 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adbc6b-11c8-4703-ad5a-828c9e823489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:21.759013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adbc6b-11c8-4703-ad5a-828c9e823489 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:21.861597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 549/1104] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:21.870710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220aecc-17ca-46ad-88fc-b4c324c8d50e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:21.988236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220aecc-17ca-46ad-88fc-b4c324c8d50e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:22.022584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07576487-f33c-4abf-8830-dc30513b78f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:22.023518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07576487-f33c-4abf-8830-dc30513b78f7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:22.023800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07576487-f33c-4abf-8830-dc30513b78f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:22.024075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07576487-f33c-4abf-8830-dc30513b78f7 None admin] PATH_INFO: `/v3/projects/58bbd3bde9d743288dd7804c0c350525` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:22.025423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07576487-f33c-4abf-8830-dc30513b78f7 None admin] RBAC: Authorizing `identity:get_project(project_id=58bbd3bde9d743288dd7804c0c350525)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:22.026853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07576487-f33c-4abf-8830-dc30513b78f7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:22.028569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 556/1105] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:31:21 2022] GET /identity/v3/projects/58bbd3bde9d743288dd7804c0c350525 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:22.035995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adbc6b-11c8-4703-ad5a-828c9e823489 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:22.145946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adbc6b-11c8-4703-ad5a-828c9e823489 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:22.180425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c2e91-3a77-4304-8ac4-0dccfaf33458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:22.181099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c2e91-3a77-4304-8ac4-0dccfaf33458 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:22.181376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c2e91-3a77-4304-8ac4-0dccfaf33458 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:22.181645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c2e91-3a77-4304-8ac4-0dccfaf33458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:22.182461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c2e91-3a77-4304-8ac4-0dccfaf33458 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=58bbd3bde9d743288dd7804c0c350525, user.id=d83fccadf4524e039f0ff1cda0089012) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:22.182767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c2e91-3a77-4304-8ac4-0dccfaf33458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:22.184184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c2e91-3a77-4304-8ac4-0dccfaf33458 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:22.189968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 550/1106] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:31:22 2022] GET /identity/v3/role_assignments?user.id=d83fccadf4524e039f0ff1cda0089012&scope.project.id=58bbd3bde9d743288dd7804c0c350525&role.id=e55ef11844a742f5889243f189bd59bc => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:24.438703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5aa82-1246-4f60-908f-ade85499d4cc None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:24.439265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5aa82-1246-4f60-908f-ade85499d4cc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:24.439265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5aa82-1246-4f60-908f-ade85499d4cc None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:24.440649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 557/1107] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:24.447180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9e418-4c54-4c03-8b2c-b13ee18c44d5 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:24.447898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9e418-4c54-4c03-8b2c-b13ee18c44d5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:24.448044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9e418-4c54-4c03-8b2c-b13ee18c44d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:24.477268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c4d9e418-4c54-4c03-8b2c-b13ee18c44d5 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:24.478446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c4d9e418-4c54-4c03-8b2c-b13ee18c44d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:24.545009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 551/1108] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:24.569868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5aa82-1246-4f60-908f-ade85499d4cc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:24.674388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5aa82-1246-4f60-908f-ade85499d4cc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:24.708468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac1e8e-fe6d-40dd-ad07-22d9b307207f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:24.709277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac1e8e-fe6d-40dd-ad07-22d9b307207f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:24.709554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac1e8e-fe6d-40dd-ad07-22d9b307207f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:24.709824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac1e8e-fe6d-40dd-ad07-22d9b307207f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:24.711222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac1e8e-fe6d-40dd-ad07-22d9b307207f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:24.712680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac1e8e-fe6d-40dd-ad07-22d9b307207f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:24.714288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 558/1109] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:24.720920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9e418-4c54-4c03-8b2c-b13ee18c44d5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:24.827980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9e418-4c54-4c03-8b2c-b13ee18c44d5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:24.862425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bff2dc-7686-497a-91ea-2123814dbb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:24.863077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bff2dc-7686-497a-91ea-2123814dbb6b None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:24.863348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bff2dc-7686-497a-91ea-2123814dbb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:24.863613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bff2dc-7686-497a-91ea-2123814dbb6b None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:24.864310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bff2dc-7686-497a-91ea-2123814dbb6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:24.865770 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bff2dc-7686-497a-91ea-2123814dbb6b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:24.892842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 552/1110] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:31:24 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:31:27.202769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad0ee2-3500-4504-912c-8ef1a0072650 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:27.203331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad0ee2-3500-4504-912c-8ef1a0072650 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:27.203331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad0ee2-3500-4504-912c-8ef1a0072650 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:27.204747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 559/1111] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:27.211137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581559a-0b9d-4f75-9b8a-b0b1eeb20efc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:27.211764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581559a-0b9d-4f75-9b8a-b0b1eeb20efc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:27.211862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581559a-0b9d-4f75-9b8a-b0b1eeb20efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:27.238815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7581559a-0b9d-4f75-9b8a-b0b1eeb20efc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:27.240749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7581559a-0b9d-4f75-9b8a-b0b1eeb20efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:27.312039 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 553/1112] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:27.345059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad0ee2-3500-4504-912c-8ef1a0072650 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:27.452956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad0ee2-3500-4504-912c-8ef1a0072650 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:27.490795 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79479d8-48b5-4fdc-8566-3075b44a029a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:27.491644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79479d8-48b5-4fdc-8566-3075b44a029a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:27.492137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79479d8-48b5-4fdc-8566-3075b44a029a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:27.492755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79479d8-48b5-4fdc-8566-3075b44a029a None admin] PATH_INFO: `/v3/roles/ec08a86158a44ee2b82951b44b228df2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:27.494340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79479d8-48b5-4fdc-8566-3075b44a029a None admin] RBAC: Authorizing `identity:get_role(role_id=ec08a86158a44ee2b82951b44b228df2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:27.495710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79479d8-48b5-4fdc-8566-3075b44a029a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:27.497112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 560/1113] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:27 2022] GET /identity/v3/roles/ec08a86158a44ee2b82951b44b228df2 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:27.503172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581559a-0b9d-4f75-9b8a-b0b1eeb20efc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:27.598948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581559a-0b9d-4f75-9b8a-b0b1eeb20efc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:27.640725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784df02-0aa5-4e53-ad1d-b67369e7c850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:27.641609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784df02-0aa5-4e53-ad1d-b67369e7c850 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:27.642014 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784df02-0aa5-4e53-ad1d-b67369e7c850 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:27.642286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784df02-0aa5-4e53-ad1d-b67369e7c850 None admin] PATH_INFO: `/v3/users/f22227f1159e4cf69eff53eab96570af` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:27.655220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784df02-0aa5-4e53-ad1d-b67369e7c850 None admin] RBAC: Authorizing `identity:get_user(user_id=f22227f1159e4cf69eff53eab96570af)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:27.656738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784df02-0aa5-4e53-ad1d-b67369e7c850 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:27.658377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 554/1114] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:27 2022] GET /identity/v3/users/f22227f1159e4cf69eff53eab96570af => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:27.664968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79479d8-48b5-4fdc-8566-3075b44a029a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:27.797020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79479d8-48b5-4fdc-8566-3075b44a029a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:27.832541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f08852-d6fd-4f63-91ce-955a5fd44227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:27.833208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f08852-d6fd-4f63-91ce-955a5fd44227 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:27.833486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f08852-d6fd-4f63-91ce-955a5fd44227 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:27.833758 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f08852-d6fd-4f63-91ce-955a5fd44227 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:27.913473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f08852-d6fd-4f63-91ce-955a5fd44227 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:27.915348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f08852-d6fd-4f63-91ce-955a5fd44227 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:28.014875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 561/1115] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:28.021727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784df02-0aa5-4e53-ad1d-b67369e7c850 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:28.142775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784df02-0aa5-4e53-ad1d-b67369e7c850 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:28.178655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c3ca3-7817-4a06-80b3-5c0cbc341da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:28.179518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c3ca3-7817-4a06-80b3-5c0cbc341da0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:28.179792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c3ca3-7817-4a06-80b3-5c0cbc341da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:28.180059 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c3ca3-7817-4a06-80b3-5c0cbc341da0 None admin] PATH_INFO: `/v3/projects/58bbd3bde9d743288dd7804c0c350525` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:28.181309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c3ca3-7817-4a06-80b3-5c0cbc341da0 None admin] RBAC: Authorizing `identity:get_project(project_id=58bbd3bde9d743288dd7804c0c350525)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:28.182775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c3ca3-7817-4a06-80b3-5c0cbc341da0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:28.184522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 555/1116] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:31:28 2022] GET /identity/v3/projects/58bbd3bde9d743288dd7804c0c350525 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:28.191010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f08852-d6fd-4f63-91ce-955a5fd44227 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:28.297116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f08852-d6fd-4f63-91ce-955a5fd44227 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:28.331158 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233bbb8-857d-4596-9572-10c3413f564d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:28.331793 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233bbb8-857d-4596-9572-10c3413f564d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:28.332074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233bbb8-857d-4596-9572-10c3413f564d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:28.332338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233bbb8-857d-4596-9572-10c3413f564d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:28.333239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233bbb8-857d-4596-9572-10c3413f564d None admin] RBAC: Adding query filter params (role.id=ec08a86158a44ee2b82951b44b228df2, scope.project.id=58bbd3bde9d743288dd7804c0c350525, user.id=f22227f1159e4cf69eff53eab96570af) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:28.333548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233bbb8-857d-4596-9572-10c3413f564d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:28.334965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233bbb8-857d-4596-9572-10c3413f564d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:28.341348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 562/1117] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:31:28 2022] GET /identity/v3/role_assignments?user.id=f22227f1159e4cf69eff53eab96570af&scope.project.id=58bbd3bde9d743288dd7804c0c350525&role.id=ec08a86158a44ee2b82951b44b228df2 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:30.586409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39e783-b44d-4ade-b480-0cd8116e0f39 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:30.586974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39e783-b44d-4ade-b480-0cd8116e0f39 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:30.586974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39e783-b44d-4ade-b480-0cd8116e0f39 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:30.588322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 556/1118] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:30.595073 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c713c6-f86f-4f5c-904f-77c05d8ed633 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:30.595815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c713c6-f86f-4f5c-904f-77c05d8ed633 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:30.595947 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c713c6-f86f-4f5c-904f-77c05d8ed633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:30.615457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e9c713c6-f86f-4f5c-904f-77c05d8ed633 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:30.616671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e9c713c6-f86f-4f5c-904f-77c05d8ed633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:30.686269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 563/1119] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:30.713332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39e783-b44d-4ade-b480-0cd8116e0f39 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:30.820934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39e783-b44d-4ade-b480-0cd8116e0f39 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:30.857269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d7cf5-7e10-4baa-b1c2-6b30b92b082f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:30.858050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d7cf5-7e10-4baa-b1c2-6b30b92b082f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:30.858331 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d7cf5-7e10-4baa-b1c2-6b30b92b082f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:30.858597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d7cf5-7e10-4baa-b1c2-6b30b92b082f None admin] PATH_INFO: `/v3/roles/ec08a86158a44ee2b82951b44b228df2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:30.860177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d7cf5-7e10-4baa-b1c2-6b30b92b082f None admin] RBAC: Authorizing `identity:get_role(role_id=ec08a86158a44ee2b82951b44b228df2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:30.861595 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d7cf5-7e10-4baa-b1c2-6b30b92b082f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:30.862973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 557/1120] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:30 2022] GET /identity/v3/roles/ec08a86158a44ee2b82951b44b228df2 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:30.869475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c713c6-f86f-4f5c-904f-77c05d8ed633 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:30.982165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c713c6-f86f-4f5c-904f-77c05d8ed633 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:31.016067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d2233-0689-4fa0-9392-e1f22fb77865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:31.016964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d2233-0689-4fa0-9392-e1f22fb77865 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:31.017245 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d2233-0689-4fa0-9392-e1f22fb77865 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:31.017518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d2233-0689-4fa0-9392-e1f22fb77865 None admin] PATH_INFO: `/v3/users/f22227f1159e4cf69eff53eab96570af` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:31.018820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d2233-0689-4fa0-9392-e1f22fb77865 None admin] RBAC: Authorizing `identity:get_user(user_id=f22227f1159e4cf69eff53eab96570af)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:31.020244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d2233-0689-4fa0-9392-e1f22fb77865 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:31.021901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 564/1121] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:30 2022] GET /identity/v3/users/f22227f1159e4cf69eff53eab96570af => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:31.032356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d7cf5-7e10-4baa-b1c2-6b30b92b082f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:31.141361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d7cf5-7e10-4baa-b1c2-6b30b92b082f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:31.188831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdc49b-d356-4dbe-9c4d-7320f882d115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:31.190810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdc49b-d356-4dbe-9c4d-7320f882d115 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:31.191450 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdc49b-d356-4dbe-9c4d-7320f882d115 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:31.192057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdc49b-d356-4dbe-9c4d-7320f882d115 None admin] PATH_INFO: `/v3/projects/58bbd3bde9d743288dd7804c0c350525` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:31.195105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdc49b-d356-4dbe-9c4d-7320f882d115 None admin] RBAC: Authorizing `identity:get_project(project_id=58bbd3bde9d743288dd7804c0c350525)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:31.196587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdc49b-d356-4dbe-9c4d-7320f882d115 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:31.198330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 558/1122] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:31:31 2022] GET /identity/v3/projects/58bbd3bde9d743288dd7804c0c350525 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:31.207819 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d2233-0689-4fa0-9392-e1f22fb77865 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:31.324483 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d2233-0689-4fa0-9392-e1f22fb77865 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:31.359326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b32b03-8e5b-4a43-b2fd-b5eae8ded7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:31.360194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b32b03-8e5b-4a43-b2fd-b5eae8ded7a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:31.360470 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b32b03-8e5b-4a43-b2fd-b5eae8ded7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:31.360760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b32b03-8e5b-4a43-b2fd-b5eae8ded7a3 None admin] PATH_INFO: `/v3/projects/58bbd3bde9d743288dd7804c0c350525/users/f22227f1159e4cf69eff53eab96570af/roles/ec08a86158a44ee2b82951b44b228df2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:31.363278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b32b03-8e5b-4a43-b2fd-b5eae8ded7a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=58bbd3bde9d743288dd7804c0c350525, user_id=f22227f1159e4cf69eff53eab96570af, role_id=ec08a86158a44ee2b82951b44b228df2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:31.364733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b32b03-8e5b-4a43-b2fd-b5eae8ded7a3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:31.380340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 565/1123] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:31:31 2022] PUT /identity/v3/projects/58bbd3bde9d743288dd7804c0c350525/users/f22227f1159e4cf69eff53eab96570af/roles/ec08a86158a44ee2b82951b44b228df2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:31:33.585154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a9050-83e5-4b81-a481-1644cc6188d3 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:33.586179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a9050-83e5-4b81-a481-1644cc6188d3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:33.586332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a9050-83e5-4b81-a481-1644cc6188d3 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:33.588475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 559/1124] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:33.597494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ee9f9-0791-4738-8942-4402147fe90f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:33.597924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ee9f9-0791-4738-8942-4402147fe90f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:33.598030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ee9f9-0791-4738-8942-4402147fe90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:33.617690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-906ee9f9-0791-4738-8942-4402147fe90f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:33.618867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-906ee9f9-0791-4738-8942-4402147fe90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:33.687876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 566/1125] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:33.713810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a9050-83e5-4b81-a481-1644cc6188d3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:33.821754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a9050-83e5-4b81-a481-1644cc6188d3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:33.855310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270da46-cf18-4d27-aaeb-20105c274a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:33.856079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270da46-cf18-4d27-aaeb-20105c274a68 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:33.856355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270da46-cf18-4d27-aaeb-20105c274a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:33.856644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270da46-cf18-4d27-aaeb-20105c274a68 None admin] PATH_INFO: `/v3/roles/ec08a86158a44ee2b82951b44b228df2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:33.858106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270da46-cf18-4d27-aaeb-20105c274a68 None admin] RBAC: Authorizing `identity:get_role(role_id=ec08a86158a44ee2b82951b44b228df2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:33.859473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270da46-cf18-4d27-aaeb-20105c274a68 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:33.860840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 560/1126] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:33 2022] GET /identity/v3/roles/ec08a86158a44ee2b82951b44b228df2 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:33.867403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ee9f9-0791-4738-8942-4402147fe90f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:33.980236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ee9f9-0791-4738-8942-4402147fe90f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:34.013878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48354dc-9508-49fb-bb31-2bfe17c9ef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:34.014725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48354dc-9508-49fb-bb31-2bfe17c9ef6c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:34.014998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48354dc-9508-49fb-bb31-2bfe17c9ef6c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:34.015262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48354dc-9508-49fb-bb31-2bfe17c9ef6c None admin] PATH_INFO: `/v3/users/f22227f1159e4cf69eff53eab96570af` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:34.016538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48354dc-9508-49fb-bb31-2bfe17c9ef6c None admin] RBAC: Authorizing `identity:get_user(user_id=f22227f1159e4cf69eff53eab96570af)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:34.017995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48354dc-9508-49fb-bb31-2bfe17c9ef6c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:34.019601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 567/1127] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:33 2022] GET /identity/v3/users/f22227f1159e4cf69eff53eab96570af => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:34.027400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270da46-cf18-4d27-aaeb-20105c274a68 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:34.147536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270da46-cf18-4d27-aaeb-20105c274a68 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:34.180916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7493b5e-54d9-4cc6-9475-deeb624fe23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:34.181542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7493b5e-54d9-4cc6-9475-deeb624fe23e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:34.181815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7493b5e-54d9-4cc6-9475-deeb624fe23e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:34.182080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7493b5e-54d9-4cc6-9475-deeb624fe23e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:34.247428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7493b5e-54d9-4cc6-9475-deeb624fe23e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:34.248887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7493b5e-54d9-4cc6-9475-deeb624fe23e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:34.353536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 561/1128] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:34.359660 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48354dc-9508-49fb-bb31-2bfe17c9ef6c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:34.457052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48354dc-9508-49fb-bb31-2bfe17c9ef6c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:34.491144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8057f-01ab-47ab-a9ef-4dae236d2194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:34.492030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8057f-01ab-47ab-a9ef-4dae236d2194 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:34.492331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8057f-01ab-47ab-a9ef-4dae236d2194 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:34.492618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8057f-01ab-47ab-a9ef-4dae236d2194 None admin] PATH_INFO: `/v3/projects/58bbd3bde9d743288dd7804c0c350525` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:34.493868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8057f-01ab-47ab-a9ef-4dae236d2194 None admin] RBAC: Authorizing `identity:get_project(project_id=58bbd3bde9d743288dd7804c0c350525)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:34.495288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8057f-01ab-47ab-a9ef-4dae236d2194 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:34.496988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 568/1129] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:31:34 2022] GET /identity/v3/projects/58bbd3bde9d743288dd7804c0c350525 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:34.503652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7493b5e-54d9-4cc6-9475-deeb624fe23e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:34.622297 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7493b5e-54d9-4cc6-9475-deeb624fe23e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:34.659252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241f93c-bdf4-46c0-963d-a6e60f139f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:34.659912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241f93c-bdf4-46c0-963d-a6e60f139f0b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:34.660198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241f93c-bdf4-46c0-963d-a6e60f139f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:34.660471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241f93c-bdf4-46c0-963d-a6e60f139f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:34.661340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241f93c-bdf4-46c0-963d-a6e60f139f0b None admin] RBAC: Adding query filter params (role.id=ec08a86158a44ee2b82951b44b228df2, scope.project.id=58bbd3bde9d743288dd7804c0c350525, user.id=f22227f1159e4cf69eff53eab96570af) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:34.661651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241f93c-bdf4-46c0-963d-a6e60f139f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:34.663087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241f93c-bdf4-46c0-963d-a6e60f139f0b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:34.670127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 562/1130] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:31:34 2022] GET /identity/v3/role_assignments?user.id=f22227f1159e4cf69eff53eab96570af&scope.project.id=58bbd3bde9d743288dd7804c0c350525&role.id=ec08a86158a44ee2b82951b44b228df2 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:36.908647 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7185fc-24c3-4cc8-9c6f-a2bd09a3d10c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:36.909495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7185fc-24c3-4cc8-9c6f-a2bd09a3d10c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:36.909495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7185fc-24c3-4cc8-9c6f-a2bd09a3d10c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:36.911509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 569/1131] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:36.918263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bb2f6-b245-47b3-a3f6-abfbfcd28d0c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:36.918963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bb2f6-b245-47b3-a3f6-abfbfcd28d0c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:36.919165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bb2f6-b245-47b3-a3f6-abfbfcd28d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:36.941587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-129bb2f6-b245-47b3-a3f6-abfbfcd28d0c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:36.942758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-129bb2f6-b245-47b3-a3f6-abfbfcd28d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:37.014974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 563/1132] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:37.042039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7185fc-24c3-4cc8-9c6f-a2bd09a3d10c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:37.145391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7185fc-24c3-4cc8-9c6f-a2bd09a3d10c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:37.178718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3c47e-2d3a-4265-a72b-1f4554ca682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:37.179468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3c47e-2d3a-4265-a72b-1f4554ca682c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:37.179753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3c47e-2d3a-4265-a72b-1f4554ca682c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:37.180106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3c47e-2d3a-4265-a72b-1f4554ca682c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:37.181334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3c47e-2d3a-4265-a72b-1f4554ca682c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:37.182761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3c47e-2d3a-4265-a72b-1f4554ca682c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:37.184386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 570/1133] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:37.190972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bb2f6-b245-47b3-a3f6-abfbfcd28d0c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:37.290368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bb2f6-b245-47b3-a3f6-abfbfcd28d0c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:37.323806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa39436-1347-41fb-9d23-518c98c32e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:37.324452 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa39436-1347-41fb-9d23-518c98c32e54 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:37.324763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa39436-1347-41fb-9d23-518c98c32e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:37.325025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa39436-1347-41fb-9d23-518c98c32e54 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:37.325717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa39436-1347-41fb-9d23-518c98c32e54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:37.327142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa39436-1347-41fb-9d23-518c98c32e54 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:37.355959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 564/1134] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:31:37 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:31:39.588990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc8892-f795-4485-9dab-25f32ad42ebf None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:39.589912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc8892-f795-4485-9dab-25f32ad42ebf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:39.589912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc8892-f795-4485-9dab-25f32ad42ebf None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:39.592451 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 571/1135] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:39.597868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd010f39-e899-4bde-867d-c6f9f0910995 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:39.598309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd010f39-e899-4bde-867d-c6f9f0910995 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:39.598395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd010f39-e899-4bde-867d-c6f9f0910995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:39.615570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-cd010f39-e899-4bde-867d-c6f9f0910995 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:39.616748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cd010f39-e899-4bde-867d-c6f9f0910995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:39.682620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 565/1136] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:39.707913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc8892-f795-4485-9dab-25f32ad42ebf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:39.822225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc8892-f795-4485-9dab-25f32ad42ebf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:39.858419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97265001-3615-4a28-8bf4-e6e8039268aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:39.859134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97265001-3615-4a28-8bf4-e6e8039268aa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:39.859412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97265001-3615-4a28-8bf4-e6e8039268aa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:39.859680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97265001-3615-4a28-8bf4-e6e8039268aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:39.860998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97265001-3615-4a28-8bf4-e6e8039268aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:39.862459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97265001-3615-4a28-8bf4-e6e8039268aa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:39.864154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 572/1137] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:31:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:39.870461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd010f39-e899-4bde-867d-c6f9f0910995 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:39.992264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd010f39-e899-4bde-867d-c6f9f0910995 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:40.032046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb54668-9670-4485-9a72-77e306580915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:40.032765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54668-9670-4485-9a72-77e306580915 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:40.033047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54668-9670-4485-9a72-77e306580915 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:40.033315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54668-9670-4485-9a72-77e306580915 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:40.034020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb54668-9670-4485-9a72-77e306580915 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:40.035464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb54668-9670-4485-9a72-77e306580915 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:40.058428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 566/1138] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:31:39 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:31:42.327368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c86193-8a1f-4e9b-84a2-b0e73fcbcdf6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:42.327968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c86193-8a1f-4e9b-84a2-b0e73fcbcdf6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:42.327968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c86193-8a1f-4e9b-84a2-b0e73fcbcdf6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:42.329334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 573/1139] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:42.335653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad5caf-48ce-455d-a875-348f98bf5ed3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:42.336738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad5caf-48ce-455d-a875-348f98bf5ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:42.336738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad5caf-48ce-455d-a875-348f98bf5ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:42.360861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-27ad5caf-48ce-455d-a875-348f98bf5ed3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:42.362052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-27ad5caf-48ce-455d-a875-348f98bf5ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:42.432850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 567/1140] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:42.461155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c86193-8a1f-4e9b-84a2-b0e73fcbcdf6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:42.568981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c86193-8a1f-4e9b-84a2-b0e73fcbcdf6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:42.605231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b996992-4483-4185-ab20-725ff2af8a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:42.605939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b996992-4483-4185-ab20-725ff2af8a58 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:42.606207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b996992-4483-4185-ab20-725ff2af8a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:42.606474 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b996992-4483-4185-ab20-725ff2af8a58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:42.616765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b996992-4483-4185-ab20-725ff2af8a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8b996992-4483-4185-ab20-725ff2af8a58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:31:42.624126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:42.625235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:42.626190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 574/1141] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:31:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:42.631573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad5caf-48ce-455d-a875-348f98bf5ed3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:42.728347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad5caf-48ce-455d-a875-348f98bf5ed3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:42.765050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dadda-be29-4af1-99c7-c75ce026a07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:42.765706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dadda-be29-4af1-99c7-c75ce026a07d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:42.765987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dadda-be29-4af1-99c7-c75ce026a07d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:42.766251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dadda-be29-4af1-99c7-c75ce026a07d None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:42.766981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04dadda-be29-4af1-99c7-c75ce026a07d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:42.767277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04dadda-be29-4af1-99c7-c75ce026a07d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:42.768664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04dadda-be29-4af1-99c7-c75ce026a07d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:42.777054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 568/1142] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:31:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:42.783098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b996992-4483-4185-ab20-725ff2af8a58 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:42.882334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b996992-4483-4185-ab20-725ff2af8a58 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:42.917852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2713f23-f561-40a9-ba6b-704bbbdd314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:42.918705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713f23-f561-40a9-ba6b-704bbbdd314d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:42.918979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713f23-f561-40a9-ba6b-704bbbdd314d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:42.919244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713f23-f561-40a9-ba6b-704bbbdd314d None admin] PATH_INFO: `/v3/users/bdb06538e378487ba499cab816b748bd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:42.933253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2713f23-f561-40a9-ba6b-704bbbdd314d None admin] RBAC: Authorizing `identity:get_user(user_id=bdb06538e378487ba499cab816b748bd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:42.934674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2713f23-f561-40a9-ba6b-704bbbdd314d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:42.936297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 575/1143] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:42 2022] GET /identity/v3/users/bdb06538e378487ba499cab816b748bd => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:42.942497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dadda-be29-4af1-99c7-c75ce026a07d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:43.056780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dadda-be29-4af1-99c7-c75ce026a07d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:43.095553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a223e3a-3100-4c42-84ae-5fa108533b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:43.096194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a223e3a-3100-4c42-84ae-5fa108533b27 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:43.096482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a223e3a-3100-4c42-84ae-5fa108533b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:43.096776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a223e3a-3100-4c42-84ae-5fa108533b27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:43.162791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a223e3a-3100-4c42-84ae-5fa108533b27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:43.164202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a223e3a-3100-4c42-84ae-5fa108533b27 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:43.259184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 569/1144] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:43.268750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2713f23-f561-40a9-ba6b-704bbbdd314d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:43.383027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2713f23-f561-40a9-ba6b-704bbbdd314d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:43.418165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe0af4-79c6-44a5-9644-05745a209603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:43.418946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe0af4-79c6-44a5-9644-05745a209603 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:43.419217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe0af4-79c6-44a5-9644-05745a209603 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:43.419483 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe0af4-79c6-44a5-9644-05745a209603 None admin] PATH_INFO: `/v3/projects/778437e1235f4920b14f5b5dbefbe7ca` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:43.420699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe0af4-79c6-44a5-9644-05745a209603 None admin] RBAC: Authorizing `identity:get_project(project_id=778437e1235f4920b14f5b5dbefbe7ca)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:43.422115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe0af4-79c6-44a5-9644-05745a209603 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:43.423797 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 576/1145] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:31:43 2022] GET /identity/v3/projects/778437e1235f4920b14f5b5dbefbe7ca => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:43.430964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a223e3a-3100-4c42-84ae-5fa108533b27 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:43.529489 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a223e3a-3100-4c42-84ae-5fa108533b27 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:43.563071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e7d2b-cc54-48f9-938f-72e87f5ec11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:43.563652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e7d2b-cc54-48f9-938f-72e87f5ec11b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:43.563935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e7d2b-cc54-48f9-938f-72e87f5ec11b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:43.564201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e7d2b-cc54-48f9-938f-72e87f5ec11b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:43.565056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e7d2b-cc54-48f9-938f-72e87f5ec11b None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=778437e1235f4920b14f5b5dbefbe7ca, user.id=bdb06538e378487ba499cab816b748bd) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:43.565363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e7d2b-cc54-48f9-938f-72e87f5ec11b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:43.566790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e7d2b-cc54-48f9-938f-72e87f5ec11b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:43.572078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 570/1146] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:31:43 2022] GET /identity/v3/role_assignments?user.id=bdb06538e378487ba499cab816b748bd&scope.project.id=778437e1235f4920b14f5b5dbefbe7ca&role.id=e55ef11844a742f5889243f189bd59bc => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:45.821111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae65aee-ef3d-4781-9c5c-1a5ba25d6af9 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:45.821953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae65aee-ef3d-4781-9c5c-1a5ba25d6af9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:45.822378 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae65aee-ef3d-4781-9c5c-1a5ba25d6af9 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:45.824576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 577/1147] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:45.833624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d74fa-b594-43f8-9472-79568fc88872 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:45.834060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d74fa-b594-43f8-9472-79568fc88872 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:45.834171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d74fa-b594-43f8-9472-79568fc88872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:45.856062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-940d74fa-b594-43f8-9472-79568fc88872 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:45.857271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-940d74fa-b594-43f8-9472-79568fc88872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:45.924560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 571/1148] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:45.949436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae65aee-ef3d-4781-9c5c-1a5ba25d6af9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:46.055761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae65aee-ef3d-4781-9c5c-1a5ba25d6af9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:46.089180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ed5cb-a08c-40eb-84de-292f77372c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:46.089944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed5cb-a08c-40eb-84de-292f77372c5c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:46.090216 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed5cb-a08c-40eb-84de-292f77372c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:46.090849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed5cb-a08c-40eb-84de-292f77372c5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:46.101211 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ed5cb-a08c-40eb-84de-292f77372c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-427ed5cb-a08c-40eb-84de-292f77372c5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:31:46.108699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:46.109871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:46.110956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 578/1149] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:31:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:46.116167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d74fa-b594-43f8-9472-79568fc88872 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:46.217409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d74fa-b594-43f8-9472-79568fc88872 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:46.256470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a76b4d-8df2-4493-a540-f73453e0af7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:46.257555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76b4d-8df2-4493-a540-f73453e0af7e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:46.257971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76b4d-8df2-4493-a540-f73453e0af7e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:46.258373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76b4d-8df2-4493-a540-f73453e0af7e None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:46.259475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a76b4d-8df2-4493-a540-f73453e0af7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:46.259778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a76b4d-8df2-4493-a540-f73453e0af7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:46.261177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a76b4d-8df2-4493-a540-f73453e0af7e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:46.269583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 572/1150] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:31:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:46.277007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ed5cb-a08c-40eb-84de-292f77372c5c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:46.404147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ed5cb-a08c-40eb-84de-292f77372c5c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:46.439921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872458f-1176-4e01-888e-cda8ad6ea8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:46.440808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872458f-1176-4e01-888e-cda8ad6ea8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:46.441084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872458f-1176-4e01-888e-cda8ad6ea8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:46.441348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872458f-1176-4e01-888e-cda8ad6ea8b5 None admin] PATH_INFO: `/v3/users/bdb06538e378487ba499cab816b748bd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:46.442638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872458f-1176-4e01-888e-cda8ad6ea8b5 None admin] RBAC: Authorizing `identity:get_user(user_id=bdb06538e378487ba499cab816b748bd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:46.444065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872458f-1176-4e01-888e-cda8ad6ea8b5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:46.445776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 579/1151] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:46 2022] GET /identity/v3/users/bdb06538e378487ba499cab816b748bd => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:46.452829 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a76b4d-8df2-4493-a540-f73453e0af7e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:46.557264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a76b4d-8df2-4493-a540-f73453e0af7e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:46.591244 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4594ada-d121-4853-85ea-dc246821e332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:46.592089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4594ada-d121-4853-85ea-dc246821e332 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:46.592368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4594ada-d121-4853-85ea-dc246821e332 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:46.592667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4594ada-d121-4853-85ea-dc246821e332 None admin] PATH_INFO: `/v3/projects/778437e1235f4920b14f5b5dbefbe7ca` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:46.593954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4594ada-d121-4853-85ea-dc246821e332 None admin] RBAC: Authorizing `identity:get_project(project_id=778437e1235f4920b14f5b5dbefbe7ca)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:46.595392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4594ada-d121-4853-85ea-dc246821e332 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:46.597062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 573/1152] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:31:46 2022] GET /identity/v3/projects/778437e1235f4920b14f5b5dbefbe7ca => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:46.605078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872458f-1176-4e01-888e-cda8ad6ea8b5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:46.723322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872458f-1176-4e01-888e-cda8ad6ea8b5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:46.756282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815677a-12ed-4bdd-8996-cf131964098f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:46.757199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815677a-12ed-4bdd-8996-cf131964098f None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:46.757476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815677a-12ed-4bdd-8996-cf131964098f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:46.757740 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815677a-12ed-4bdd-8996-cf131964098f None admin] PATH_INFO: `/v3/projects/778437e1235f4920b14f5b5dbefbe7ca/users/bdb06538e378487ba499cab816b748bd/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:46.759825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815677a-12ed-4bdd-8996-cf131964098f None admin] RBAC: Authorizing `identity:create_grant(project_id=778437e1235f4920b14f5b5dbefbe7ca, user_id=bdb06538e378487ba499cab816b748bd, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:46.761277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815677a-12ed-4bdd-8996-cf131964098f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:46.777326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 580/1153] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:31:46 2022] PUT /identity/v3/projects/778437e1235f4920b14f5b5dbefbe7ca/users/bdb06538e378487ba499cab816b748bd/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:31:48.971198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31964c16-9e46-4b0e-b2a9-1da828653844 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:48.971775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31964c16-9e46-4b0e-b2a9-1da828653844 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:48.971775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31964c16-9e46-4b0e-b2a9-1da828653844 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:48.973138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 574/1154] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:48.979459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379eab3d-159a-4c01-a73c-4ddf1fc54b44 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:48.980190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379eab3d-159a-4c01-a73c-4ddf1fc54b44 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:48.980357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379eab3d-159a-4c01-a73c-4ddf1fc54b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:49.002758 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-379eab3d-159a-4c01-a73c-4ddf1fc54b44 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:49.003946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-379eab3d-159a-4c01-a73c-4ddf1fc54b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:49.080413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 581/1155] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:49.104275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31964c16-9e46-4b0e-b2a9-1da828653844 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:49.210322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31964c16-9e46-4b0e-b2a9-1da828653844 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:49.244305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c32dce-beee-46bc-9c2c-9b9b65a2b1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:49.245060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c32dce-beee-46bc-9c2c-9b9b65a2b1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:49.245337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c32dce-beee-46bc-9c2c-9b9b65a2b1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:49.245605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c32dce-beee-46bc-9c2c-9b9b65a2b1d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:49.256058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c32dce-beee-46bc-9c2c-9b9b65a2b1d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-06c32dce-beee-46bc-9c2c-9b9b65a2b1d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:31:49.263725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:49.264914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:31:49.266120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 575/1156] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:49.271174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379eab3d-159a-4c01-a73c-4ddf1fc54b44 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:49.383687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379eab3d-159a-4c01-a73c-4ddf1fc54b44 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:49.417640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43159db9-7ea3-412f-97db-168256cfa0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:49.418297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43159db9-7ea3-412f-97db-168256cfa0ca None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:49.418574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43159db9-7ea3-412f-97db-168256cfa0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:49.418839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43159db9-7ea3-412f-97db-168256cfa0ca None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:49.419576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43159db9-7ea3-412f-97db-168256cfa0ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:49.419882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43159db9-7ea3-412f-97db-168256cfa0ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:49.421281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43159db9-7ea3-412f-97db-168256cfa0ca None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:49.429284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 582/1157] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:31:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:49.438112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c32dce-beee-46bc-9c2c-9b9b65a2b1d8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:49.541442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c32dce-beee-46bc-9c2c-9b9b65a2b1d8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:49.574668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae5473-a32a-47b0-81e9-0a3a82df94b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:49.575529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5473-a32a-47b0-81e9-0a3a82df94b3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:49.575802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5473-a32a-47b0-81e9-0a3a82df94b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:49.576065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5473-a32a-47b0-81e9-0a3a82df94b3 None admin] PATH_INFO: `/v3/users/bdb06538e378487ba499cab816b748bd` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:49.577349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae5473-a32a-47b0-81e9-0a3a82df94b3 None admin] RBAC: Authorizing `identity:get_user(user_id=bdb06538e378487ba499cab816b748bd)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:49.578787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae5473-a32a-47b0-81e9-0a3a82df94b3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:49.580398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 576/1158] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:31:49 2022] GET /identity/v3/users/bdb06538e378487ba499cab816b748bd => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:49.589412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43159db9-7ea3-412f-97db-168256cfa0ca None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:49.695907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43159db9-7ea3-412f-97db-168256cfa0ca None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:49.729726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209a2b4-98da-47ae-bd65-ee24028fc78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:49.730372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209a2b4-98da-47ae-bd65-ee24028fc78a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:49.730650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209a2b4-98da-47ae-bd65-ee24028fc78a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:49.730918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209a2b4-98da-47ae-bd65-ee24028fc78a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:49.797886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209a2b4-98da-47ae-bd65-ee24028fc78a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:49.799261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209a2b4-98da-47ae-bd65-ee24028fc78a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:49.894938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 583/1159] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:49.904057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae5473-a32a-47b0-81e9-0a3a82df94b3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:50.000505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae5473-a32a-47b0-81e9-0a3a82df94b3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:50.035353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bf2cf-192e-4116-9592-954bdbe9a692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:50.036212 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bf2cf-192e-4116-9592-954bdbe9a692 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:50.036506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bf2cf-192e-4116-9592-954bdbe9a692 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:50.036864 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bf2cf-192e-4116-9592-954bdbe9a692 None admin] PATH_INFO: `/v3/projects/778437e1235f4920b14f5b5dbefbe7ca` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:50.038166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bf2cf-192e-4116-9592-954bdbe9a692 None admin] RBAC: Authorizing `identity:get_project(project_id=778437e1235f4920b14f5b5dbefbe7ca)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:50.039596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bf2cf-192e-4116-9592-954bdbe9a692 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:50.041458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 577/1160] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:31:49 2022] GET /identity/v3/projects/778437e1235f4920b14f5b5dbefbe7ca => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:31:50.047792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209a2b4-98da-47ae-bd65-ee24028fc78a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:50.145881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209a2b4-98da-47ae-bd65-ee24028fc78a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:50.181432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5faed-d3a0-4c20-a972-5f44204cb094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:50.182073 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5faed-d3a0-4c20-a972-5f44204cb094 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:50.182348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5faed-d3a0-4c20-a972-5f44204cb094 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:50.182619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5faed-d3a0-4c20-a972-5f44204cb094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:50.183432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5faed-d3a0-4c20-a972-5f44204cb094 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=778437e1235f4920b14f5b5dbefbe7ca, user.id=bdb06538e378487ba499cab816b748bd) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:50.183735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5faed-d3a0-4c20-a972-5f44204cb094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:50.185202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5faed-d3a0-4c20-a972-5f44204cb094 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:50.191493 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 584/1161] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:31:50 2022] GET /identity/v3/role_assignments?user.id=bdb06538e378487ba499cab816b748bd&scope.project.id=778437e1235f4920b14f5b5dbefbe7ca&role.id=e55ef11844a742f5889243f189bd59bc => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:52.518344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30147176-0fab-4a4a-95c2-262383cfa6f5 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:52.519209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30147176-0fab-4a4a-95c2-262383cfa6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:52.519209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30147176-0fab-4a4a-95c2-262383cfa6f5 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:52.521475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 578/1162] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:52.528119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda8438-b763-4d5d-98fb-5d401dfd40d3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:52.528881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda8438-b763-4d5d-98fb-5d401dfd40d3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:52.529029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda8438-b763-4d5d-98fb-5d401dfd40d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:52.556447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2dda8438-b763-4d5d-98fb-5d401dfd40d3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:52.557663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2dda8438-b763-4d5d-98fb-5d401dfd40d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:52.626106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 585/1163] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:52.649006 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30147176-0fab-4a4a-95c2-262383cfa6f5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:52.775600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30147176-0fab-4a4a-95c2-262383cfa6f5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:52.812754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a604d-bfaf-430c-9e6c-32c93ca54ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:52.813396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a604d-bfaf-430c-9e6c-32c93ca54ecf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:52.813670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a604d-bfaf-430c-9e6c-32c93ca54ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:52.813945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a604d-bfaf-430c-9e6c-32c93ca54ecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:52.882200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a604d-bfaf-430c-9e6c-32c93ca54ecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:52.883696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a604d-bfaf-430c-9e6c-32c93ca54ecf None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:52.995446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 579/1164] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:31:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:31:53.002450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda8438-b763-4d5d-98fb-5d401dfd40d3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:53.107219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda8438-b763-4d5d-98fb-5d401dfd40d3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:53.141004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a81ee7-e452-491a-8e8f-dd6ded1a93ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:53.141787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a81ee7-e452-491a-8e8f-dd6ded1a93ec None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:53.142064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a81ee7-e452-491a-8e8f-dd6ded1a93ec None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:53.142337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a81ee7-e452-491a-8e8f-dd6ded1a93ec None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:53.148796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a81ee7-e452-491a-8e8f-dd6ded1a93ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:53.150287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a81ee7-e452-491a-8e8f-dd6ded1a93ec None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a1a81ee7-e452-491a-8e8f-dd6ded1a93ec None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:31:53.163871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:31:53.165074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:31:53.166171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 586/1165] 10.222.0.50 () {56 vars in 1117 bytes} [Fri Apr 29 18:31:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:31:53.174819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a604d-bfaf-430c-9e6c-32c93ca54ecf None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:53.283433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a604d-bfaf-430c-9e6c-32c93ca54ecf None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:53.320096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762425f-2a29-4249-9426-16d38b29eda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:53.320781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762425f-2a29-4249-9426-16d38b29eda4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:53.321094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762425f-2a29-4249-9426-16d38b29eda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:53.321357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762425f-2a29-4249-9426-16d38b29eda4 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:53.322259 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762425f-2a29-4249-9426-16d38b29eda4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:31:53.322562 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762425f-2a29-4249-9426-16d38b29eda4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:53.323925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762425f-2a29-4249-9426-16d38b29eda4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:53.331109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 580/1166] 10.222.0.50 () {56 vars in 1115 bytes} [Fri Apr 29 18:31:53 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:55.674811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc8d3f-5be0-4877-807d-455edf40f1e7 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:55.675643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc8d3f-5be0-4877-807d-455edf40f1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:55.675643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc8d3f-5be0-4877-807d-455edf40f1e7 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:55.677943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 587/1167] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:55.683581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f353d2e-3a7a-4990-bce3-d89383a51c92 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:55.684027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f353d2e-3a7a-4990-bce3-d89383a51c92 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:55.684126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f353d2e-3a7a-4990-bce3-d89383a51c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:55.702266 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1f353d2e-3a7a-4990-bce3-d89383a51c92 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:55.703419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1f353d2e-3a7a-4990-bce3-d89383a51c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:55.769718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 581/1168] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:55.793154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc8d3f-5be0-4877-807d-455edf40f1e7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:55.918045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc8d3f-5be0-4877-807d-455edf40f1e7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:55.958283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11513b-d470-45cb-9f96-f0e4745fb5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:55.959248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11513b-d470-45cb-9f96-f0e4745fb5dc None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:55.959661 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11513b-d470-45cb-9f96-f0e4745fb5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:55.960083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11513b-d470-45cb-9f96-f0e4745fb5dc None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:55.961301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11513b-d470-45cb-9f96-f0e4745fb5dc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:55.963932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11513b-d470-45cb-9f96-f0e4745fb5dc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:55.984506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 588/1169] 10.222.0.50 () {60 vars in 1138 bytes} [Fri Apr 29 18:31:55 2022] POST /identity/v3/domains => generated 267 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:31:58.223029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a60514-d0e4-46cb-8491-6a3fd8df1c76 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:58.223620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a60514-d0e4-46cb-8491-6a3fd8df1c76 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:58.223620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a60514-d0e4-46cb-8491-6a3fd8df1c76 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:58.224958 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 582/1170] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:31:58.231463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad21199-0ac5-492f-9af2-c0713ad24e3a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:58.231897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad21199-0ac5-492f-9af2-c0713ad24e3a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:58.232038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad21199-0ac5-492f-9af2-c0713ad24e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:58.259677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0ad21199-0ac5-492f-9af2-c0713ad24e3a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:31:58.260886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0ad21199-0ac5-492f-9af2-c0713ad24e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:31:58.327345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 589/1171] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:31:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:31:58.351450 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a60514-d0e4-46cb-8491-6a3fd8df1c76 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:58.462444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a60514-d0e4-46cb-8491-6a3fd8df1c76 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:58.500594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d731d0-6f9f-49f2-9cec-27653488aefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:58.501405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d731d0-6f9f-49f2-9cec-27653488aefa None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:58.501683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d731d0-6f9f-49f2-9cec-27653488aefa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:58.501949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d731d0-6f9f-49f2-9cec-27653488aefa None admin] PATH_INFO: `/v3/domains/e8764901e1154fe2a92541a6271a25ba` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:58.512360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d731d0-6f9f-49f2-9cec-27653488aefa None admin] RBAC: Authorizing `identity:get_domain(domain_id=e8764901e1154fe2a92541a6271a25ba)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:58.513832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d731d0-6f9f-49f2-9cec-27653488aefa None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:58.515434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 583/1172] 10.222.0.50 () {56 vars in 1183 bytes} [Fri Apr 29 18:31:58 2022] GET /identity/v3/domains/e8764901e1154fe2a92541a6271a25ba => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:31:58.525664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad21199-0ac5-492f-9af2-c0713ad24e3a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:31:58.634408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad21199-0ac5-492f-9af2-c0713ad24e3a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:31:58.666885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e650bc-34da-41c8-9743-b646401df196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:31:58.667536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e650bc-34da-41c8-9743-b646401df196 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:31:58.667812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e650bc-34da-41c8-9743-b646401df196 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:31:58.668081 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e650bc-34da-41c8-9743-b646401df196 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:31:58.668816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e650bc-34da-41c8-9743-b646401df196 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:31:58.670248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e650bc-34da-41c8-9743-b646401df196 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:31:58.704449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 590/1173] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:31:58 2022] POST /identity/v3/projects => generated 361 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:32:00.935858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143f3e5-3653-431f-b6b7-f41b41ea8ee8 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:00.936406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143f3e5-3653-431f-b6b7-f41b41ea8ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:00.936406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143f3e5-3653-431f-b6b7-f41b41ea8ee8 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:00.937691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 584/1174] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:32:00.943118 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58773b-e5a3-49bb-8265-79041f4470c0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:00.943538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58773b-e5a3-49bb-8265-79041f4470c0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:00.943647 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58773b-e5a3-49bb-8265-79041f4470c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:00.962031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2e58773b-e5a3-49bb-8265-79041f4470c0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:32:00.963182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2e58773b-e5a3-49bb-8265-79041f4470c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:32:01.028830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 591/1175] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:32:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:32:01.054440 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143f3e5-3653-431f-b6b7-f41b41ea8ee8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:01.164311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143f3e5-3653-431f-b6b7-f41b41ea8ee8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:01.198502 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76186dd4-e87c-4c1c-a033-7c8f1ab5a14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:01.199289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186dd4-e87c-4c1c-a033-7c8f1ab5a14b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:01.199568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186dd4-e87c-4c1c-a033-7c8f1ab5a14b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:01.199839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186dd4-e87c-4c1c-a033-7c8f1ab5a14b None admin] PATH_INFO: `/v3/domains/e8764901e1154fe2a92541a6271a25ba` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:01.201148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76186dd4-e87c-4c1c-a033-7c8f1ab5a14b None admin] RBAC: Authorizing `identity:get_domain(domain_id=e8764901e1154fe2a92541a6271a25ba)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:01.202588 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76186dd4-e87c-4c1c-a033-7c8f1ab5a14b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:01.204176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 585/1176] 10.222.0.50 () {56 vars in 1183 bytes} [Fri Apr 29 18:32:01 2022] GET /identity/v3/domains/e8764901e1154fe2a92541a6271a25ba => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:32:01.213644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58773b-e5a3-49bb-8265-79041f4470c0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:01.323936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58773b-e5a3-49bb-8265-79041f4470c0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:01.358867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fb60a-3abc-408e-b4c9-7df31a281351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:01.359525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fb60a-3abc-408e-b4c9-7df31a281351 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:01.359798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fb60a-3abc-408e-b4c9-7df31a281351 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:01.360066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fb60a-3abc-408e-b4c9-7df31a281351 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:01.360803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fb60a-3abc-408e-b4c9-7df31a281351 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:01.362227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fb60a-3abc-408e-b4c9-7df31a281351 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:01.386779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 592/1177] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:32:01 2022] POST /identity/v3/users => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:32:03.667877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18b27d-44c1-473e-ba74-df94ed9757a4 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:03.668787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18b27d-44c1-473e-ba74-df94ed9757a4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:03.668923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18b27d-44c1-473e-ba74-df94ed9757a4 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:03.671115 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 586/1178] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:32:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:32:03.676698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1e5b0-4527-427d-84c5-82a4507620ed None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:03.677132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1e5b0-4527-427d-84c5-82a4507620ed None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:03.677229 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1e5b0-4527-427d-84c5-82a4507620ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:03.694339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cab1e5b0-4527-427d-84c5-82a4507620ed None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:32:03.695511 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cab1e5b0-4527-427d-84c5-82a4507620ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:32:03.761367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 593/1179] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:32:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:32:03.785757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18b27d-44c1-473e-ba74-df94ed9757a4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:03.894206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18b27d-44c1-473e-ba74-df94ed9757a4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:03.928219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc71f7-6e58-4d73-9196-fa4c6704a1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:03.929041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc71f7-6e58-4d73-9196-fa4c6704a1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:03.929327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc71f7-6e58-4d73-9196-fa4c6704a1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:03.929596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc71f7-6e58-4d73-9196-fa4c6704a1f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:03.940878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc71f7-6e58-4d73-9196-fa4c6704a1f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-dedc71f7-6e58-4d73-9196-fa4c6704a1f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:32:03.949258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:32:03.950603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:32:03.951649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 587/1180] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:32:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:32:03.957131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1e5b0-4527-427d-84c5-82a4507620ed None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:04.064968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1e5b0-4527-427d-84c5-82a4507620ed None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:04.098754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62887a0-4e96-4975-a352-d69db567deac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:04.099406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62887a0-4e96-4975-a352-d69db567deac None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:04.099685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62887a0-4e96-4975-a352-d69db567deac None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:04.099949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62887a0-4e96-4975-a352-d69db567deac None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:04.100719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62887a0-4e96-4975-a352-d69db567deac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:32:04.101032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62887a0-4e96-4975-a352-d69db567deac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:04.102461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62887a0-4e96-4975-a352-d69db567deac None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:04.110389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 594/1181] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:32:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:32:04.119878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc71f7-6e58-4d73-9196-fa4c6704a1f1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:04.223698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc71f7-6e58-4d73-9196-fa4c6704a1f1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:04.256994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374531a-b25a-4a9b-89d6-178e6cb885c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:04.257846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374531a-b25a-4a9b-89d6-178e6cb885c8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:04.258124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374531a-b25a-4a9b-89d6-178e6cb885c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:04.258435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374531a-b25a-4a9b-89d6-178e6cb885c8 None admin] PATH_INFO: `/v3/users/9b6b68d056d644fb991294ca7b934867` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:04.271561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374531a-b25a-4a9b-89d6-178e6cb885c8 None admin] RBAC: Authorizing `identity:get_user(user_id=9b6b68d056d644fb991294ca7b934867)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:04.273026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374531a-b25a-4a9b-89d6-178e6cb885c8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:04.274717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 588/1182] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:32:04 2022] GET /identity/v3/users/9b6b68d056d644fb991294ca7b934867 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:32:04.280885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62887a0-4e96-4975-a352-d69db567deac None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:04.385250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62887a0-4e96-4975-a352-d69db567deac None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:04.419785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f9771-7824-4768-a89f-74d2606291af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:04.420415 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f9771-7824-4768-a89f-74d2606291af None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:04.420711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f9771-7824-4768-a89f-74d2606291af None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:04.420979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f9771-7824-4768-a89f-74d2606291af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:04.486174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f9771-7824-4768-a89f-74d2606291af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:04.487527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f9771-7824-4768-a89f-74d2606291af None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:04.588002 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 595/1183] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:32:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:32:04.594414 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374531a-b25a-4a9b-89d6-178e6cb885c8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:04.692965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374531a-b25a-4a9b-89d6-178e6cb885c8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:04.727004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a2fa6-2ce3-424b-ba36-469729f46098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:04.727853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a2fa6-2ce3-424b-ba36-469729f46098 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:04.728126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a2fa6-2ce3-424b-ba36-469729f46098 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:04.728394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a2fa6-2ce3-424b-ba36-469729f46098 None admin] PATH_INFO: `/v3/projects/1ec1fc7ffe3948fc8e3a3fc0ae5519ed` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:04.729692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a2fa6-2ce3-424b-ba36-469729f46098 None admin] RBAC: Authorizing `identity:get_project(project_id=1ec1fc7ffe3948fc8e3a3fc0ae5519ed)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:04.731108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a2fa6-2ce3-424b-ba36-469729f46098 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:04.732788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 589/1184] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:32:04 2022] GET /identity/v3/projects/1ec1fc7ffe3948fc8e3a3fc0ae5519ed => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:32:04.739805 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f9771-7824-4768-a89f-74d2606291af None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:04.846289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f9771-7824-4768-a89f-74d2606291af None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:04.879734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91d4fa-1459-41ca-a441-6f132d02bae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:04.880359 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91d4fa-1459-41ca-a441-6f132d02bae7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:04.880667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91d4fa-1459-41ca-a441-6f132d02bae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:04.880933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91d4fa-1459-41ca-a441-6f132d02bae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:04.881741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91d4fa-1459-41ca-a441-6f132d02bae7 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=1ec1fc7ffe3948fc8e3a3fc0ae5519ed, user.id=9b6b68d056d644fb991294ca7b934867) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:32:04.882039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91d4fa-1459-41ca-a441-6f132d02bae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:04.883522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91d4fa-1459-41ca-a441-6f132d02bae7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:04.888782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 596/1185] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:32:04 2022] GET /identity/v3/role_assignments?user.id=9b6b68d056d644fb991294ca7b934867&scope.project.id=1ec1fc7ffe3948fc8e3a3fc0ae5519ed&role.id=e55ef11844a742f5889243f189bd59bc => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:32:07.078463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc4bd-93a0-4cd9-840e-f3c7ba6691e0 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:07.079318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc4bd-93a0-4cd9-840e-f3c7ba6691e0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:07.079318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc4bd-93a0-4cd9-840e-f3c7ba6691e0 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:07.081545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 590/1186] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:32:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:32:07.090827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d1678-c150-44ea-9116-db01d78bb454 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:07.091259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d1678-c150-44ea-9116-db01d78bb454 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:07.091397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d1678-c150-44ea-9116-db01d78bb454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:07.112576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-597d1678-c150-44ea-9116-db01d78bb454 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:32:07.113769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-597d1678-c150-44ea-9116-db01d78bb454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:32:07.183996 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 597/1187] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:32:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:32:07.210990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fc4bd-93a0-4cd9-840e-f3c7ba6691e0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:07.317612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fc4bd-93a0-4cd9-840e-f3c7ba6691e0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:07.363769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbf860-3c95-4c82-9531-5ec0e72aea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:07.364525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbf860-3c95-4c82-9531-5ec0e72aea82 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:07.364802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbf860-3c95-4c82-9531-5ec0e72aea82 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:07.365074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbf860-3c95-4c82-9531-5ec0e72aea82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:07.376711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbf860-3c95-4c82-9531-5ec0e72aea82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-b2fbf860-3c95-4c82-9531-5ec0e72aea82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:32:07.384252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:32:07.385344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:32:07.386366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 591/1188] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:32:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:32:07.393624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d1678-c150-44ea-9116-db01d78bb454 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:07.492231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d1678-c150-44ea-9116-db01d78bb454 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:07.537029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a65843-f9df-4e73-8f17-8adfc9299b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:07.537948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a65843-f9df-4e73-8f17-8adfc9299b25 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:07.538300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a65843-f9df-4e73-8f17-8adfc9299b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:07.538642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a65843-f9df-4e73-8f17-8adfc9299b25 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:07.539704 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a65843-f9df-4e73-8f17-8adfc9299b25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:32:07.540101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a65843-f9df-4e73-8f17-8adfc9299b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:07.542250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a65843-f9df-4e73-8f17-8adfc9299b25 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:07.555701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 598/1189] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:32:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:32:07.565536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbf860-3c95-4c82-9531-5ec0e72aea82 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:07.679092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbf860-3c95-4c82-9531-5ec0e72aea82 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:07.713553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe302d-de69-49ac-b82b-68cd6f0dc137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:07.714424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe302d-de69-49ac-b82b-68cd6f0dc137 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:07.714703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe302d-de69-49ac-b82b-68cd6f0dc137 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:07.715355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe302d-de69-49ac-b82b-68cd6f0dc137 None admin] PATH_INFO: `/v3/users/9b6b68d056d644fb991294ca7b934867` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:07.716700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe302d-de69-49ac-b82b-68cd6f0dc137 None admin] RBAC: Authorizing `identity:get_user(user_id=9b6b68d056d644fb991294ca7b934867)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:07.718145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe302d-de69-49ac-b82b-68cd6f0dc137 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:07.719779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 592/1190] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:32:07 2022] GET /identity/v3/users/9b6b68d056d644fb991294ca7b934867 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:32:07.726445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a65843-f9df-4e73-8f17-8adfc9299b25 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:07.828887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a65843-f9df-4e73-8f17-8adfc9299b25 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:07.862427 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f5e24-238b-45d1-a1d3-0aafbc317c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:07.863263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f5e24-238b-45d1-a1d3-0aafbc317c9a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:07.863544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f5e24-238b-45d1-a1d3-0aafbc317c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:07.863816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f5e24-238b-45d1-a1d3-0aafbc317c9a None admin] PATH_INFO: `/v3/projects/1ec1fc7ffe3948fc8e3a3fc0ae5519ed` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:07.865064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f5e24-238b-45d1-a1d3-0aafbc317c9a None admin] RBAC: Authorizing `identity:get_project(project_id=1ec1fc7ffe3948fc8e3a3fc0ae5519ed)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:07.866489 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f5e24-238b-45d1-a1d3-0aafbc317c9a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:07.868313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 599/1191] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:32:07 2022] GET /identity/v3/projects/1ec1fc7ffe3948fc8e3a3fc0ae5519ed => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:32:07.875028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe302d-de69-49ac-b82b-68cd6f0dc137 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:07.989953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe302d-de69-49ac-b82b-68cd6f0dc137 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:08.026093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc9d9e-b9fd-4b5b-81b8-d580390b3165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:08.026984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc9d9e-b9fd-4b5b-81b8-d580390b3165 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:08.027266 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc9d9e-b9fd-4b5b-81b8-d580390b3165 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:08.027543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc9d9e-b9fd-4b5b-81b8-d580390b3165 None admin] PATH_INFO: `/v3/projects/1ec1fc7ffe3948fc8e3a3fc0ae5519ed/users/9b6b68d056d644fb991294ca7b934867/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:08.029779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc9d9e-b9fd-4b5b-81b8-d580390b3165 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ec1fc7ffe3948fc8e3a3fc0ae5519ed, user_id=9b6b68d056d644fb991294ca7b934867, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:08.031224 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc9d9e-b9fd-4b5b-81b8-d580390b3165 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:08.047066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 593/1192] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:32:07 2022] PUT /identity/v3/projects/1ec1fc7ffe3948fc8e3a3fc0ae5519ed/users/9b6b68d056d644fb991294ca7b934867/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:32:10.249631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5c7f4-a34c-492e-bb93-2aa048a6e068 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:10.250208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5c7f4-a34c-492e-bb93-2aa048a6e068 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:10.250208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5c7f4-a34c-492e-bb93-2aa048a6e068 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:10.252409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 600/1193] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:32:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:32:10.260976 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a82d00-e5f0-4b6b-aeb1-9615bd65314f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:10.261378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a82d00-e5f0-4b6b-aeb1-9615bd65314f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:10.261567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a82d00-e5f0-4b6b-aeb1-9615bd65314f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:10.289705 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f3a82d00-e5f0-4b6b-aeb1-9615bd65314f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:32:10.290891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f3a82d00-e5f0-4b6b-aeb1-9615bd65314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:32:10.359506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 594/1194] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:32:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:32:10.393407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5c7f4-a34c-492e-bb93-2aa048a6e068 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:10.529815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5c7f4-a34c-492e-bb93-2aa048a6e068 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:10.564009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253c2d4-cbf0-4898-87c6-2e6b824d57a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:10.564836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253c2d4-cbf0-4898-87c6-2e6b824d57a2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:10.565106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253c2d4-cbf0-4898-87c6-2e6b824d57a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:10.565375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253c2d4-cbf0-4898-87c6-2e6b824d57a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:10.575459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253c2d4-cbf0-4898-87c6-2e6b824d57a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3253c2d4-cbf0-4898-87c6-2e6b824d57a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:32:10.582856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:32:10.583939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:32:10.585022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 601/1195] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:32:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:32:10.590871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a82d00-e5f0-4b6b-aeb1-9615bd65314f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:10.690956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a82d00-e5f0-4b6b-aeb1-9615bd65314f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:10.725169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad270e1-816d-4fc4-9a5c-d3a9f0351a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:10.725817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad270e1-816d-4fc4-9a5c-d3a9f0351a01 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:10.726090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad270e1-816d-4fc4-9a5c-d3a9f0351a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:10.726358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad270e1-816d-4fc4-9a5c-d3a9f0351a01 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:10.727127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad270e1-816d-4fc4-9a5c-d3a9f0351a01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:32:10.727428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad270e1-816d-4fc4-9a5c-d3a9f0351a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:10.728818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad270e1-816d-4fc4-9a5c-d3a9f0351a01 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:10.736673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 595/1196] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:32:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:32:10.742707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253c2d4-cbf0-4898-87c6-2e6b824d57a2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:10.850796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253c2d4-cbf0-4898-87c6-2e6b824d57a2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:10.884763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd9323-cab5-48e8-a88d-a89db12f4d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:10.885627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd9323-cab5-48e8-a88d-a89db12f4d43 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:10.885905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd9323-cab5-48e8-a88d-a89db12f4d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:10.886151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd9323-cab5-48e8-a88d-a89db12f4d43 None admin] PATH_INFO: `/v3/users/9b6b68d056d644fb991294ca7b934867` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:10.887482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd9323-cab5-48e8-a88d-a89db12f4d43 None admin] RBAC: Authorizing `identity:get_user(user_id=9b6b68d056d644fb991294ca7b934867)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:10.888959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd9323-cab5-48e8-a88d-a89db12f4d43 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:10.890577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 602/1197] 10.222.0.50 () {56 vars in 1177 bytes} [Fri Apr 29 18:32:10 2022] GET /identity/v3/users/9b6b68d056d644fb991294ca7b934867 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:32:10.897266 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad270e1-816d-4fc4-9a5c-d3a9f0351a01 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:11.023110 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad270e1-816d-4fc4-9a5c-d3a9f0351a01 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:11.056588 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba94823-0e9e-4665-90af-70223863bdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:11.057231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba94823-0e9e-4665-90af-70223863bdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:11.057512 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba94823-0e9e-4665-90af-70223863bdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:11.057779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba94823-0e9e-4665-90af-70223863bdc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:11.123379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba94823-0e9e-4665-90af-70223863bdc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:11.124778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba94823-0e9e-4665-90af-70223863bdc5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:11.224062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 596/1198] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:32:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:32:11.234169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd9323-cab5-48e8-a88d-a89db12f4d43 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:11.338814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd9323-cab5-48e8-a88d-a89db12f4d43 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:11.372735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940ba67-9f49-4ad6-befa-f0499eff7984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:11.373673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940ba67-9f49-4ad6-befa-f0499eff7984 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:11.373862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940ba67-9f49-4ad6-befa-f0499eff7984 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:11.374150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940ba67-9f49-4ad6-befa-f0499eff7984 None admin] PATH_INFO: `/v3/projects/1ec1fc7ffe3948fc8e3a3fc0ae5519ed` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:11.375444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940ba67-9f49-4ad6-befa-f0499eff7984 None admin] RBAC: Authorizing `identity:get_project(project_id=1ec1fc7ffe3948fc8e3a3fc0ae5519ed)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:11.376914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940ba67-9f49-4ad6-befa-f0499eff7984 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:11.378584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 603/1199] 10.222.0.50 () {56 vars in 1186 bytes} [Fri Apr 29 18:32:11 2022] GET /identity/v3/projects/1ec1fc7ffe3948fc8e3a3fc0ae5519ed => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:32:11.385728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba94823-0e9e-4665-90af-70223863bdc5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:32:11.483763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba94823-0e9e-4665-90af-70223863bdc5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:32:11.517428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f296e-7cf4-4ff2-be7d-657e7b48d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:32:11.518071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f296e-7cf4-4ff2-be7d-657e7b48d3ea None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:32:11.518346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f296e-7cf4-4ff2-be7d-657e7b48d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:32:11.518615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f296e-7cf4-4ff2-be7d-657e7b48d3ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:32:11.519430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f296e-7cf4-4ff2-be7d-657e7b48d3ea None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=1ec1fc7ffe3948fc8e3a3fc0ae5519ed, user.id=9b6b68d056d644fb991294ca7b934867) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:32:11.519741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f296e-7cf4-4ff2-be7d-657e7b48d3ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:32:11.521212 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f296e-7cf4-4ff2-be7d-657e7b48d3ea None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:32:11.527007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 597/1200] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:32:11 2022] GET /identity/v3/role_assignments?user.id=9b6b68d056d644fb991294ca7b934867&scope.project.id=1ec1fc7ffe3948fc8e3a3fc0ae5519ed&role.id=e55ef11844a742f5889243f189bd59bc => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:13.043136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567e170-6dda-4ab5-90f7-3b4c5731db2f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:13.043764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567e170-6dda-4ab5-90f7-3b4c5731db2f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:13.043764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567e170-6dda-4ab5-90f7-3b4c5731db2f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:13.045170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 604/1201] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:13.051745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77b169-20ac-470a-a23d-fab0b057603c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:13.052301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77b169-20ac-470a-a23d-fab0b057603c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:13.052301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77b169-20ac-470a-a23d-fab0b057603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:13.072413 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ce77b169-20ac-470a-a23d-fab0b057603c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:13.073728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ce77b169-20ac-470a-a23d-fab0b057603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:13.154090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 598/1202] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:33:13.181093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567e170-6dda-4ab5-90f7-3b4c5731db2f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:13.307984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567e170-6dda-4ab5-90f7-3b4c5731db2f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:13.358601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b764af-c216-4226-9428-10a464347e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:13.359418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b764af-c216-4226-9428-10a464347e1a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:13.359680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b764af-c216-4226-9428-10a464347e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:13.359945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b764af-c216-4226-9428-10a464347e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:13.371601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b764af-c216-4226-9428-10a464347e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:13.373135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b764af-c216-4226-9428-10a464347e1a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-92b764af-c216-4226-9428-10a464347e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:13.392562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:13.393939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:13.394925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:13.394925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:13.394925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:13.394925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:13.394925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 605/1203] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:33:13.399994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77b169-20ac-470a-a23d-fab0b057603c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:13.519643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77b169-20ac-470a-a23d-fab0b057603c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:13.558977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf08e90-ce57-461b-91b8-f33a773bf7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:13.559789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf08e90-ce57-461b-91b8-f33a773bf7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:13.560167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf08e90-ce57-461b-91b8-f33a773bf7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:13.560575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf08e90-ce57-461b-91b8-f33a773bf7a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:13.561444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf08e90-ce57-461b-91b8-f33a773bf7a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:13.561857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf08e90-ce57-461b-91b8-f33a773bf7a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:13.563446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf08e90-ce57-461b-91b8-f33a773bf7a3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:13.576543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 599/1204] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:33:13.582316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b764af-c216-4226-9428-10a464347e1a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:13.734205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b764af-c216-4226-9428-10a464347e1a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:13.780693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f096812-e8b9-4ec2-809b-15b7427845c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:13.782156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f096812-e8b9-4ec2-809b-15b7427845c2 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:13.782424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f096812-e8b9-4ec2-809b-15b7427845c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:13.782674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f096812-e8b9-4ec2-809b-15b7427845c2 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:13.784128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f096812-e8b9-4ec2-809b-15b7427845c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:13.785779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f096812-e8b9-4ec2-809b-15b7427845c2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:13.825113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 606/1205] 10.222.0.50 () {60 vars in 1132 bytes} [Fri Apr 29 18:33:13 2022] POST /identity/v3/users => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:33:16.293501 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70b717-36ba-4ddf-ad0a-0945490ddcda None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:16.294060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70b717-36ba-4ddf-ad0a-0945490ddcda None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:16.294060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70b717-36ba-4ddf-ad0a-0945490ddcda None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:16.295737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 600/1206] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:16.301985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b199233-4400-4360-b3d7-a47912fe85f8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:16.302550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b199233-4400-4360-b3d7-a47912fe85f8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:16.302550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b199233-4400-4360-b3d7-a47912fe85f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:16.322085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0b199233-4400-4360-b3d7-a47912fe85f8 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:16.323495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0b199233-4400-4360-b3d7-a47912fe85f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:16.404627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 607/1207] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:33:16.430477 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70b717-36ba-4ddf-ad0a-0945490ddcda None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:16.543913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70b717-36ba-4ddf-ad0a-0945490ddcda None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:16.586296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc12f69-0bec-4590-bd66-b8437cd31dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:16.587218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc12f69-0bec-4590-bd66-b8437cd31dec None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:16.587567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc12f69-0bec-4590-bd66-b8437cd31dec None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:16.587823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc12f69-0bec-4590-bd66-b8437cd31dec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:16.601635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc12f69-0bec-4590-bd66-b8437cd31dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-abc12f69-0bec-4590-bd66-b8437cd31dec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:33:16.610195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:16.611325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:16.612199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 601/1208] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:33:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:16.621329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b199233-4400-4360-b3d7-a47912fe85f8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:16.736614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b199233-4400-4360-b3d7-a47912fe85f8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:16.772902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa5be3-8adb-4318-a5c8-e96ad3981251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:16.773653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa5be3-8adb-4318-a5c8-e96ad3981251 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:16.773909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa5be3-8adb-4318-a5c8-e96ad3981251 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:16.774180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa5be3-8adb-4318-a5c8-e96ad3981251 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:16.774941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa5be3-8adb-4318-a5c8-e96ad3981251 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:16.775229 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa5be3-8adb-4318-a5c8-e96ad3981251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:16.776639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa5be3-8adb-4318-a5c8-e96ad3981251 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:16.785723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 608/1209] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:33:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:16.791474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc12f69-0bec-4590-bd66-b8437cd31dec None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:16.898941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc12f69-0bec-4590-bd66-b8437cd31dec None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:16.937823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630ee87-1b62-4711-98e8-afc65fc612c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:16.938586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630ee87-1b62-4711-98e8-afc65fc612c6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:16.938869 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630ee87-1b62-4711-98e8-afc65fc612c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:16.940763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630ee87-1b62-4711-98e8-afc65fc612c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:16.952796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630ee87-1b62-4711-98e8-afc65fc612c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:16.954434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630ee87-1b62-4711-98e8-afc65fc612c6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-1630ee87-1b62-4711-98e8-afc65fc612c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:16.973961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:16.974967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:16.976187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:16.976187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:16.976187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:16.976187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:16.980033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 602/1210] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:16.981194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa5be3-8adb-4318-a5c8-e96ad3981251 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:17.094839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa5be3-8adb-4318-a5c8-e96ad3981251 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:17.132974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732581-62cd-4382-a94f-10dd401a75c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:17.133753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732581-62cd-4382-a94f-10dd401a75c9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:17.134060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732581-62cd-4382-a94f-10dd401a75c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:17.134406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732581-62cd-4382-a94f-10dd401a75c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:17.135319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732581-62cd-4382-a94f-10dd401a75c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:17.135779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732581-62cd-4382-a94f-10dd401a75c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:17.137450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732581-62cd-4382-a94f-10dd401a75c9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:17.158138 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 609/1211] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:17.159356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630ee87-1b62-4711-98e8-afc65fc612c6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:17.268591 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630ee87-1b62-4711-98e8-afc65fc612c6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:17.302760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebf38d-8346-4efa-a06c-70e183981708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:17.303673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebf38d-8346-4efa-a06c-70e183981708 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:17.303963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebf38d-8346-4efa-a06c-70e183981708 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:17.304313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebf38d-8346-4efa-a06c-70e183981708 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:17.312705 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebf38d-8346-4efa-a06c-70e183981708 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:17.314187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebf38d-8346-4efa-a06c-70e183981708 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-1aebf38d-8346-4efa-a06c-70e183981708 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:33:17.332648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:17.340263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:17.341887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 603/1212] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:17.342497 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732581-62cd-4382-a94f-10dd401a75c9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:17.455703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732581-62cd-4382-a94f-10dd401a75c9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:17.492858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043e61b-0309-4cfd-a76b-35227618011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:17.493533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e61b-0309-4cfd-a76b-35227618011f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:17.493860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e61b-0309-4cfd-a76b-35227618011f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:17.494086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e61b-0309-4cfd-a76b-35227618011f None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:17.494901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043e61b-0309-4cfd-a76b-35227618011f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:17.495208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043e61b-0309-4cfd-a76b-35227618011f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:17.496649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043e61b-0309-4cfd-a76b-35227618011f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:17.519249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 610/1213] 10.222.0.50 () {56 vars in 1143 bytes} [Fri Apr 29 18:33:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:33:17.520573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebf38d-8346-4efa-a06c-70e183981708 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:17.628368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebf38d-8346-4efa-a06c-70e183981708 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:17.663030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b755bd-7c66-4bac-ba68-e0f49f047183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:17.663783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b755bd-7c66-4bac-ba68-e0f49f047183 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:17.664127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b755bd-7c66-4bac-ba68-e0f49f047183 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:17.664416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b755bd-7c66-4bac-ba68-e0f49f047183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:17.737524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b755bd-7c66-4bac-ba68-e0f49f047183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:17.738975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b755bd-7c66-4bac-ba68-e0f49f047183 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:17.844925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 604/1214] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:33:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:33:17.850823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043e61b-0309-4cfd-a76b-35227618011f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:17.952150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043e61b-0309-4cfd-a76b-35227618011f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:17.991961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec1707-10bd-4c2e-954b-564909a95fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:17.992740 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec1707-10bd-4c2e-954b-564909a95fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:17.993011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec1707-10bd-4c2e-954b-564909a95fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:17.993282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec1707-10bd-4c2e-954b-564909a95fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:18.004744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec1707-10bd-4c2e-954b-564909a95fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:18.006290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec1707-10bd-4c2e-954b-564909a95fe5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d3ec1707-10bd-4c2e-954b-564909a95fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:18.026053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:18.027066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:18.033031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:18.033031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:18.033031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:18.033031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:18.033031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 611/1215] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:33:18.033373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b755bd-7c66-4bac-ba68-e0f49f047183 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:18.136176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b755bd-7c66-4bac-ba68-e0f49f047183 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:18.171592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d0c3b-17f9-456a-a60a-6132e17908b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:18.172297 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d0c3b-17f9-456a-a60a-6132e17908b3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:18.172602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d0c3b-17f9-456a-a60a-6132e17908b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:18.172876 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d0c3b-17f9-456a-a60a-6132e17908b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:18.173599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d0c3b-17f9-456a-a60a-6132e17908b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:18.173901 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d0c3b-17f9-456a-a60a-6132e17908b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:18.175313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d0c3b-17f9-456a-a60a-6132e17908b3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:18.187821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 605/1216] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:18.193239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec1707-10bd-4c2e-954b-564909a95fe5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:18.295272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec1707-10bd-4c2e-954b-564909a95fe5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:18.328861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ec333-4493-4311-ae8e-91d623df31d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:18.329753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ec333-4493-4311-ae8e-91d623df31d0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:18.330092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ec333-4493-4311-ae8e-91d623df31d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:18.330324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ec333-4493-4311-ae8e-91d623df31d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:18.337422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ec333-4493-4311-ae8e-91d623df31d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:18.339226 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ec333-4493-4311-ae8e-91d623df31d0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-568ec333-4493-4311-ae8e-91d623df31d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:33:18.352926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:18.354435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:18.355418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:33:18.355418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:33:18.355418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:33:18.355418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:33:18.355418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:33:18.355418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:33:18.355418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:18.355418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:18.355418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 612/1217] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:18.361915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d0c3b-17f9-456a-a60a-6132e17908b3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:18.461500 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d0c3b-17f9-456a-a60a-6132e17908b3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:18.496404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522dba7-75ef-43e0-a07d-c0029b04f5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:18.497119 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522dba7-75ef-43e0-a07d-c0029b04f5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:18.497445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522dba7-75ef-43e0-a07d-c0029b04f5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:18.497718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522dba7-75ef-43e0-a07d-c0029b04f5d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:18.498527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522dba7-75ef-43e0-a07d-c0029b04f5d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:18.498831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522dba7-75ef-43e0-a07d-c0029b04f5d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:18.500273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522dba7-75ef-43e0-a07d-c0029b04f5d7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:18.513107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 606/1218] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:33:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:18.519007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ec333-4493-4311-ae8e-91d623df31d0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:18.627062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ec333-4493-4311-ae8e-91d623df31d0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:18.665172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61610a5a-8cbc-40cd-8a14-5a3861c23e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:18.665965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61610a5a-8cbc-40cd-8a14-5a3861c23e26 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:18.666266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61610a5a-8cbc-40cd-8a14-5a3861c23e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:18.666568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61610a5a-8cbc-40cd-8a14-5a3861c23e26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:18.667459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61610a5a-8cbc-40cd-8a14-5a3861c23e26 None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=6e934c8e7dd0425cac68eb91e5e2c0ae) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:18.667789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61610a5a-8cbc-40cd-8a14-5a3861c23e26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:18.669347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61610a5a-8cbc-40cd-8a14-5a3861c23e26 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:18.675751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 613/1219] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:33:18 2022] GET /identity/v3/role_assignments?user.id=6e934c8e7dd0425cac68eb91e5e2c0ae&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:33:21.043098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c78d2b-5473-4df7-a400-86a02db8f85d None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:21.043683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c78d2b-5473-4df7-a400-86a02db8f85d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:21.043683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c78d2b-5473-4df7-a400-86a02db8f85d None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:21.045016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 607/1220] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:21.050548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb4f2e-9177-449d-9e18-a26860317d9b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:21.051000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb4f2e-9177-449d-9e18-a26860317d9b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:21.051259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb4f2e-9177-449d-9e18-a26860317d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:21.067988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d3bb4f2e-9177-449d-9e18-a26860317d9b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:21.069450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d3bb4f2e-9177-449d-9e18-a26860317d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:21.140013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 614/1221] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:33:21.166526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c78d2b-5473-4df7-a400-86a02db8f85d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:21.284349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c78d2b-5473-4df7-a400-86a02db8f85d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:21.322323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10233f-c47c-46de-b1ca-c4f189f49e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:21.323572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10233f-c47c-46de-b1ca-c4f189f49e96 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:21.324086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10233f-c47c-46de-b1ca-c4f189f49e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:21.324686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10233f-c47c-46de-b1ca-c4f189f49e96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:21.341672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10233f-c47c-46de-b1ca-c4f189f49e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2d10233f-c47c-46de-b1ca-c4f189f49e96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:33:21.353652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:21.355553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:21.356597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 608/1222] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:33:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:21.362064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb4f2e-9177-449d-9e18-a26860317d9b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:21.466630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb4f2e-9177-449d-9e18-a26860317d9b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:21.502233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7939e-4f1f-4c69-9a51-7f4f2cd19685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:21.503049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7939e-4f1f-4c69-9a51-7f4f2cd19685 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:21.503334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7939e-4f1f-4c69-9a51-7f4f2cd19685 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:21.503609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7939e-4f1f-4c69-9a51-7f4f2cd19685 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:21.504409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7939e-4f1f-4c69-9a51-7f4f2cd19685 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:21.504704 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7939e-4f1f-4c69-9a51-7f4f2cd19685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:21.506234 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7939e-4f1f-4c69-9a51-7f4f2cd19685 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:21.515185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 615/1223] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:33:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:21.521517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10233f-c47c-46de-b1ca-c4f189f49e96 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:21.633678 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10233f-c47c-46de-b1ca-c4f189f49e96 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:21.669133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3931b-2d6e-4f8c-a076-3c36d10fc1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:21.670106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3931b-2d6e-4f8c-a076-3c36d10fc1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:21.670406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3931b-2d6e-4f8c-a076-3c36d10fc1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:21.670709 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3931b-2d6e-4f8c-a076-3c36d10fc1a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:21.682764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3931b-2d6e-4f8c-a076-3c36d10fc1a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:21.684370 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3931b-2d6e-4f8c-a076-3c36d10fc1a1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-b9d3931b-2d6e-4f8c-a076-3c36d10fc1a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:21.703105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:21.704136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:21.705094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:21.705094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:21.705094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:21.705094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:21.705094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 609/1224] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:21.711401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7939e-4f1f-4c69-9a51-7f4f2cd19685 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:21.826079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7939e-4f1f-4c69-9a51-7f4f2cd19685 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:21.861934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55add875-9742-41fb-b218-90e30f8eaee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:21.862610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add875-9742-41fb-b218-90e30f8eaee7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:21.862909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add875-9742-41fb-b218-90e30f8eaee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:21.863221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55add875-9742-41fb-b218-90e30f8eaee7 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:21.863978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55add875-9742-41fb-b218-90e30f8eaee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:21.864284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55add875-9742-41fb-b218-90e30f8eaee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:21.866163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55add875-9742-41fb-b218-90e30f8eaee7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:21.878479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 616/1225] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:21.884685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3931b-2d6e-4f8c-a076-3c36d10fc1a1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:22.009499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3931b-2d6e-4f8c-a076-3c36d10fc1a1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:22.046936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b85c73-3004-4564-8cc2-e39d94346354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:22.047755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b85c73-3004-4564-8cc2-e39d94346354 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:22.048029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b85c73-3004-4564-8cc2-e39d94346354 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:22.048294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b85c73-3004-4564-8cc2-e39d94346354 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:22.056484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b85c73-3004-4564-8cc2-e39d94346354 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:22.058073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b85c73-3004-4564-8cc2-e39d94346354 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-30b85c73-3004-4564-8cc2-e39d94346354 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:33:22.072151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:22.073488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:22.074914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 610/1226] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:22.079802 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55add875-9742-41fb-b218-90e30f8eaee7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:22.187072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55add875-9742-41fb-b218-90e30f8eaee7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:22.221049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e900f-e2a8-4591-9b33-6c7392dd8b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:22.221718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e900f-e2a8-4591-9b33-6c7392dd8b19 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:22.222016 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e900f-e2a8-4591-9b33-6c7392dd8b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:22.222262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e900f-e2a8-4591-9b33-6c7392dd8b19 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:22.223101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e900f-e2a8-4591-9b33-6c7392dd8b19 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:22.223388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e900f-e2a8-4591-9b33-6c7392dd8b19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:22.224892 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e900f-e2a8-4591-9b33-6c7392dd8b19 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:22.243283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 617/1227] 10.222.0.50 () {56 vars in 1143 bytes} [Fri Apr 29 18:33:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:33:22.250651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b85c73-3004-4564-8cc2-e39d94346354 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:22.354887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b85c73-3004-4564-8cc2-e39d94346354 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:22.389281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314ca15-5052-4da7-bd4b-9e03ddf4f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:22.389996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314ca15-5052-4da7-bd4b-9e03ddf4f89e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:22.390269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314ca15-5052-4da7-bd4b-9e03ddf4f89e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:22.390574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314ca15-5052-4da7-bd4b-9e03ddf4f89e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:22.401568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314ca15-5052-4da7-bd4b-9e03ddf4f89e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:22.403161 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314ca15-5052-4da7-bd4b-9e03ddf4f89e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5314ca15-5052-4da7-bd4b-9e03ddf4f89e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:22.421513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:22.422675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:22.423837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:22.423837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:22.423837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:22.423837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:22.423837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 611/1228] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:22.429288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e900f-e2a8-4591-9b33-6c7392dd8b19 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:22.534256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e900f-e2a8-4591-9b33-6c7392dd8b19 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:22.570306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a83efb-9ad6-43f6-a99e-fe03dfdaf4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:22.570973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a83efb-9ad6-43f6-a99e-fe03dfdaf4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:22.571252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a83efb-9ad6-43f6-a99e-fe03dfdaf4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:22.571524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a83efb-9ad6-43f6-a99e-fe03dfdaf4b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:22.572247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a83efb-9ad6-43f6-a99e-fe03dfdaf4b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:22.572600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a83efb-9ad6-43f6-a99e-fe03dfdaf4b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:22.574142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a83efb-9ad6-43f6-a99e-fe03dfdaf4b7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:22.586922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 618/1229] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:22.593757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314ca15-5052-4da7-bd4b-9e03ddf4f89e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:22.703076 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314ca15-5052-4da7-bd4b-9e03ddf4f89e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:22.738002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34536df2-6c4e-46d5-a31e-79644c611eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:22.739171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536df2-6c4e-46d5-a31e-79644c611eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:22.739599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536df2-6c4e-46d5-a31e-79644c611eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:22.740003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536df2-6c4e-46d5-a31e-79644c611eb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:22.749345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34536df2-6c4e-46d5-a31e-79644c611eb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:22.750084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34536df2-6c4e-46d5-a31e-79644c611eb7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-34536df2-6c4e-46d5-a31e-79644c611eb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:33:22.766469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:22.767570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:22.769397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:33:22.769397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:33:22.769397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:33:22.769397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:33:22.769397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:33:22.769397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:33:22.769397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:22.769397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:22.769397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 612/1230] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:33:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:22.774345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a83efb-9ad6-43f6-a99e-fe03dfdaf4b7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:22.876179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a83efb-9ad6-43f6-a99e-fe03dfdaf4b7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:22.910409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc6693-9996-4d4f-8ab7-21d0f1015f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:22.911146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc6693-9996-4d4f-8ab7-21d0f1015f62 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:22.911413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc6693-9996-4d4f-8ab7-21d0f1015f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:22.911688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc6693-9996-4d4f-8ab7-21d0f1015f62 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:22.912553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc6693-9996-4d4f-8ab7-21d0f1015f62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:22.912865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc6693-9996-4d4f-8ab7-21d0f1015f62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:22.914579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc6693-9996-4d4f-8ab7-21d0f1015f62 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:22.928279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 619/1231] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:33:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:22.935286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34536df2-6c4e-46d5-a31e-79644c611eb7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:23.049075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34536df2-6c4e-46d5-a31e-79644c611eb7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:23.083578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f6d1a-c1a5-498e-9295-3cef89dac0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:23.084774 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f6d1a-c1a5-498e-9295-3cef89dac0eb None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:23.085082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f6d1a-c1a5-498e-9295-3cef89dac0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:23.085360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f6d1a-c1a5-498e-9295-3cef89dac0eb None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/6e934c8e7dd0425cac68eb91e5e2c0ae/roles/4243cffaa7424ff99b0e23fc4940bae5` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:23.099661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f6d1a-c1a5-498e-9295-3cef89dac0eb None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=6e934c8e7dd0425cac68eb91e5e2c0ae, role_id=4243cffaa7424ff99b0e23fc4940bae5)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:23.101747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f6d1a-c1a5-498e-9295-3cef89dac0eb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:23.114573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 613/1232] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:33:22 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/6e934c8e7dd0425cac68eb91e5e2c0ae/roles/4243cffaa7424ff99b0e23fc4940bae5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:33:25.432222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8342d-bbfc-4ee8-a048-ae667720dfc7 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:25.432905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8342d-bbfc-4ee8-a048-ae667720dfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:25.433032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8342d-bbfc-4ee8-a048-ae667720dfc7 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:25.434500 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 620/1233] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:25.439938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27721b52-8610-4dda-a27c-273d90bf9cf4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:25.440398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27721b52-8610-4dda-a27c-273d90bf9cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:25.440529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27721b52-8610-4dda-a27c-273d90bf9cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:25.457547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-27721b52-8610-4dda-a27c-273d90bf9cf4 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:25.458710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-27721b52-8610-4dda-a27c-273d90bf9cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:25.528844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 614/1234] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:33:25.555307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8342d-bbfc-4ee8-a048-ae667720dfc7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:25.660657 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8342d-bbfc-4ee8-a048-ae667720dfc7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:25.694576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70457ac4-5d46-44d4-84c0-30dd1973bdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:25.695334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457ac4-5d46-44d4-84c0-30dd1973bdee None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:25.695613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457ac4-5d46-44d4-84c0-30dd1973bdee None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:25.695880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70457ac4-5d46-44d4-84c0-30dd1973bdee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:25.705921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70457ac4-5d46-44d4-84c0-30dd1973bdee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-70457ac4-5d46-44d4-84c0-30dd1973bdee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:33:25.714248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:25.715486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:25.716346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 621/1235] 10.222.0.50 () {56 vars in 1102 bytes} [Fri Apr 29 18:33:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:25.721507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27721b52-8610-4dda-a27c-273d90bf9cf4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:25.826799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27721b52-8610-4dda-a27c-273d90bf9cf4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:25.871423 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642df73-4db7-455f-8d2b-afd2a6614a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:25.872240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642df73-4db7-455f-8d2b-afd2a6614a69 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:25.872620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642df73-4db7-455f-8d2b-afd2a6614a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:25.872944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642df73-4db7-455f-8d2b-afd2a6614a69 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:25.873785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642df73-4db7-455f-8d2b-afd2a6614a69 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:25.874145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642df73-4db7-455f-8d2b-afd2a6614a69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:25.875621 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642df73-4db7-455f-8d2b-afd2a6614a69 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:25.885794 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 615/1236] 10.222.0.50 () {56 vars in 1103 bytes} [Fri Apr 29 18:33:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:25.892027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70457ac4-5d46-44d4-84c0-30dd1973bdee None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:26.005725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70457ac4-5d46-44d4-84c0-30dd1973bdee None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:26.044809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3418f7a-fdc1-4ff7-baf6-167f5860ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:26.045617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3418f7a-fdc1-4ff7-baf6-167f5860ec04 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:26.045898 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3418f7a-fdc1-4ff7-baf6-167f5860ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:26.046170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3418f7a-fdc1-4ff7-baf6-167f5860ec04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:26.064038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3418f7a-fdc1-4ff7-baf6-167f5860ec04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:26.065790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3418f7a-fdc1-4ff7-baf6-167f5860ec04 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e3418f7a-fdc1-4ff7-baf6-167f5860ec04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:26.085636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:26.086658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:26.087576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:26.087576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:26.087576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:26.087576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:26.094373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 622/1237] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Apr 29 18:33:26.100365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642df73-4db7-455f-8d2b-afd2a6614a69 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:26.205477 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642df73-4db7-455f-8d2b-afd2a6614a69 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:26.242219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea58a9c-5ff0-4d37-8f38-0305233454c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:26.242219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea58a9c-5ff0-4d37-8f38-0305233454c3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:26.242219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea58a9c-5ff0-4d37-8f38-0305233454c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:26.242219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea58a9c-5ff0-4d37-8f38-0305233454c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:26.242665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea58a9c-5ff0-4d37-8f38-0305233454c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:26.243013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea58a9c-5ff0-4d37-8f38-0305233454c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:26.244606 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea58a9c-5ff0-4d37-8f38-0305233454c3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:26.257836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 616/1238] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:26.263731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3418f7a-fdc1-4ff7-baf6-167f5860ec04 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:26.366043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3418f7a-fdc1-4ff7-baf6-167f5860ec04 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:26.399880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3c04c-2660-4986-8833-3a3c73fedc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:26.401011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3c04c-2660-4986-8833-3a3c73fedc32 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:26.401313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3c04c-2660-4986-8833-3a3c73fedc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:26.401656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3c04c-2660-4986-8833-3a3c73fedc32 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:26.410509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3c04c-2660-4986-8833-3a3c73fedc32 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:26.412163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3c04c-2660-4986-8833-3a3c73fedc32 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-15f3c04c-2660-4986-8833-3a3c73fedc32 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:33:26.429152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:26.430176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:26.431130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 623/1239] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:26.437047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea58a9c-5ff0-4d37-8f38-0305233454c3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:26.541341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea58a9c-5ff0-4d37-8f38-0305233454c3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:26.577563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb699ea5-e3ff-4645-89dd-d8138fcd5638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:26.578243 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb699ea5-e3ff-4645-89dd-d8138fcd5638 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:26.578531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb699ea5-e3ff-4645-89dd-d8138fcd5638 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:26.578784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb699ea5-e3ff-4645-89dd-d8138fcd5638 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:26.579610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb699ea5-e3ff-4645-89dd-d8138fcd5638 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:26.579909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb699ea5-e3ff-4645-89dd-d8138fcd5638 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:26.581358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb699ea5-e3ff-4645-89dd-d8138fcd5638 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:26.600264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 617/1240] 10.222.0.50 () {56 vars in 1143 bytes} [Fri Apr 29 18:33:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:26.606138 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3c04c-2660-4986-8833-3a3c73fedc32 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:26.711107 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3c04c-2660-4986-8833-3a3c73fedc32 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:26.746210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7bfbc-6a55-46d0-b6a9-4e0d20cdf517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:26.746858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7bfbc-6a55-46d0-b6a9-4e0d20cdf517 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:26.747135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7bfbc-6a55-46d0-b6a9-4e0d20cdf517 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:26.747401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7bfbc-6a55-46d0-b6a9-4e0d20cdf517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:26.815213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7bfbc-6a55-46d0-b6a9-4e0d20cdf517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:26.816884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7bfbc-6a55-46d0-b6a9-4e0d20cdf517 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:26.921636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 624/1241] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:33:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Apr 29 18:33:26.927566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb699ea5-e3ff-4645-89dd-d8138fcd5638 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:27.034594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb699ea5-e3ff-4645-89dd-d8138fcd5638 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:27.069548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb0b6-2a8f-4f66-81ef-2e47e98808bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:27.070283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb0b6-2a8f-4f66-81ef-2e47e98808bd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:27.070564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb0b6-2a8f-4f66-81ef-2e47e98808bd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:27.070834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeb0b6-2a8f-4f66-81ef-2e47e98808bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:27.082251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbeb0b6-2a8f-4f66-81ef-2e47e98808bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:27.083707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbeb0b6-2a8f-4f66-81ef-2e47e98808bd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5fbeb0b6-2a8f-4f66-81ef-2e47e98808bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:27.104356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:27.105593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:27.106970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:27.106970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:27.106970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:27.106970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:27.106970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 618/1242] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:27.111683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7bfbc-6a55-46d0-b6a9-4e0d20cdf517 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:27.219488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7bfbc-6a55-46d0-b6a9-4e0d20cdf517 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:27.259313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796b5a9-c784-4e56-beef-78fa107c1f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:27.260000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796b5a9-c784-4e56-beef-78fa107c1f85 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:27.260326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796b5a9-c784-4e56-beef-78fa107c1f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:27.260625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796b5a9-c784-4e56-beef-78fa107c1f85 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:27.261492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796b5a9-c784-4e56-beef-78fa107c1f85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:27.261834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796b5a9-c784-4e56-beef-78fa107c1f85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:27.263434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796b5a9-c784-4e56-beef-78fa107c1f85 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:27.276422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 625/1243] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:33:27.282623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb0b6-2a8f-4f66-81ef-2e47e98808bd None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:27.390427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeb0b6-2a8f-4f66-81ef-2e47e98808bd None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:27.427411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebbb5f-f4e0-4436-b12e-dc3ef0b154dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:27.428673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebbb5f-f4e0-4436-b12e-dc3ef0b154dc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:27.428971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebbb5f-f4e0-4436-b12e-dc3ef0b154dc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:27.429336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebbb5f-f4e0-4436-b12e-dc3ef0b154dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:27.437303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ebbb5f-f4e0-4436-b12e-dc3ef0b154dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:27.438866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ebbb5f-f4e0-4436-b12e-dc3ef0b154dc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-32ebbb5f-f4e0-4436-b12e-dc3ef0b154dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:33:27.452743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:27.453717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:27.454625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:33:27.454625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:33:27.454625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:33:27.454625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:33:27.454625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:33:27.454625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:33:27.454625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:27.454625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:27.454625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 619/1244] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:33:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:27.460608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796b5a9-c784-4e56-beef-78fa107c1f85 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:27.568629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796b5a9-c784-4e56-beef-78fa107c1f85 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:27.606778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af4afc-a06c-4364-8f72-e0628bd03bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:27.607594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4afc-a06c-4364-8f72-e0628bd03bed None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:27.607882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4afc-a06c-4364-8f72-e0628bd03bed None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:27.608171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4afc-a06c-4364-8f72-e0628bd03bed None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:27.609123 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af4afc-a06c-4364-8f72-e0628bd03bed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:27.609457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af4afc-a06c-4364-8f72-e0628bd03bed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:27.610999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af4afc-a06c-4364-8f72-e0628bd03bed None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:27.624740 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 626/1245] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:27.630935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebbb5f-f4e0-4436-b12e-dc3ef0b154dc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:27.748480 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebbb5f-f4e0-4436-b12e-dc3ef0b154dc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:27.785668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cdeda-e5ba-458f-a5a5-9271afaf2876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:27.786347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cdeda-e5ba-458f-a5a5-9271afaf2876 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:27.786628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cdeda-e5ba-458f-a5a5-9271afaf2876 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:27.786899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cdeda-e5ba-458f-a5a5-9271afaf2876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:27.787823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cdeda-e5ba-458f-a5a5-9271afaf2876 None admin] RBAC: Adding query filter params (role.id=4243cffaa7424ff99b0e23fc4940bae5, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=6e934c8e7dd0425cac68eb91e5e2c0ae) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:27.788270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cdeda-e5ba-458f-a5a5-9271afaf2876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:27.789743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cdeda-e5ba-458f-a5a5-9271afaf2876 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:27.795824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 620/1246] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:33:27 2022] GET /identity/v3/role_assignments?user.id=6e934c8e7dd0425cac68eb91e5e2c0ae&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=4243cffaa7424ff99b0e23fc4940bae5 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:30.132753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3cf38-2248-4876-861a-01f9f0594d17 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:30.133386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3cf38-2248-4876-861a-01f9f0594d17 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:30.133386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3cf38-2248-4876-861a-01f9f0594d17 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:30.134738 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 627/1247] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:30.140754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca399e1-3235-4fdd-9750-ff11e573df3c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:30.141275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca399e1-3235-4fdd-9750-ff11e573df3c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:30.141275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca399e1-3235-4fdd-9750-ff11e573df3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:30.160655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7ca399e1-3235-4fdd-9750-ff11e573df3c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:30.160655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7ca399e1-3235-4fdd-9750-ff11e573df3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:30.229265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 621/1248] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:33:30.253101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3cf38-2248-4876-861a-01f9f0594d17 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:30.366569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3cf38-2248-4876-861a-01f9f0594d17 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:30.409106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f281b72-9470-4923-92a4-03c5822796d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:30.409991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f281b72-9470-4923-92a4-03c5822796d4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:30.410360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f281b72-9470-4923-92a4-03c5822796d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:30.410665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f281b72-9470-4923-92a4-03c5822796d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:30.422566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f281b72-9470-4923-92a4-03c5822796d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1f281b72-9470-4923-92a4-03c5822796d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:33:30.430430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:30.431675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:30.432549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 628/1249] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:33:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:30.438229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca399e1-3235-4fdd-9750-ff11e573df3c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:30.542589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca399e1-3235-4fdd-9750-ff11e573df3c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:30.579168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a20ad-0468-4d80-98cf-e9e310e0234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:30.580359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a20ad-0468-4d80-98cf-e9e310e0234d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:30.580694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a20ad-0468-4d80-98cf-e9e310e0234d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:30.581062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a20ad-0468-4d80-98cf-e9e310e0234d None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:30.581919 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a20ad-0468-4d80-98cf-e9e310e0234d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:30.582398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a20ad-0468-4d80-98cf-e9e310e0234d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:30.584364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a20ad-0468-4d80-98cf-e9e310e0234d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:30.594257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 622/1250] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:33:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:30.600633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f281b72-9470-4923-92a4-03c5822796d4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:30.714295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f281b72-9470-4923-92a4-03c5822796d4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:30.751719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd22d1-d4b1-439a-822c-7c7e0ce08ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:30.752676 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd22d1-d4b1-439a-822c-7c7e0ce08ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:30.753014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd22d1-d4b1-439a-822c-7c7e0ce08ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:30.753280 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd22d1-d4b1-439a-822c-7c7e0ce08ed4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:30.765635 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd22d1-d4b1-439a-822c-7c7e0ce08ed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:30.767202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd22d1-d4b1-439a-822c-7c7e0ce08ed4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-39bd22d1-d4b1-439a-822c-7c7e0ce08ed4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:30.786653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:30.787774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:30.788728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:30.788728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:30.788728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:30.788728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:30.788728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 629/1251] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:30.794281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a20ad-0468-4d80-98cf-e9e310e0234d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:30.895714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a20ad-0468-4d80-98cf-e9e310e0234d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:30.929603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3324c19-1c28-4fb2-a3f2-6ef9bec1e8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:30.930380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3324c19-1c28-4fb2-a3f2-6ef9bec1e8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:30.930681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3324c19-1c28-4fb2-a3f2-6ef9bec1e8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:30.930962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3324c19-1c28-4fb2-a3f2-6ef9bec1e8e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:30.931664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3324c19-1c28-4fb2-a3f2-6ef9bec1e8e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:30.931945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3324c19-1c28-4fb2-a3f2-6ef9bec1e8e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:30.933508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3324c19-1c28-4fb2-a3f2-6ef9bec1e8e7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:30.945889 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 623/1252] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:30.952210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd22d1-d4b1-439a-822c-7c7e0ce08ed4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:31.084738 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd22d1-d4b1-439a-822c-7c7e0ce08ed4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:31.118446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a757a-d1bc-4194-9a88-13a6c29db9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:31.119248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a757a-d1bc-4194-9a88-13a6c29db9fc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:31.119542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a757a-d1bc-4194-9a88-13a6c29db9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:31.119795 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a757a-d1bc-4194-9a88-13a6c29db9fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:31.127260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a757a-d1bc-4194-9a88-13a6c29db9fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:31.128751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a757a-d1bc-4194-9a88-13a6c29db9fc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0a7a757a-d1bc-4194-9a88-13a6c29db9fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:33:31.144113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:31.145597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:31.146587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 630/1253] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:31.151768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3324c19-1c28-4fb2-a3f2-6ef9bec1e8e7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:31.257647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3324c19-1c28-4fb2-a3f2-6ef9bec1e8e7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:31.298079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6698b2-3e4d-48bc-9fb3-f8b9317a8c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:31.298756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6698b2-3e4d-48bc-9fb3-f8b9317a8c9f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:31.299036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6698b2-3e4d-48bc-9fb3-f8b9317a8c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:31.299311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6698b2-3e4d-48bc-9fb3-f8b9317a8c9f None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:31.300141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6698b2-3e4d-48bc-9fb3-f8b9317a8c9f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:31.300441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6698b2-3e4d-48bc-9fb3-f8b9317a8c9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:31.301902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6698b2-3e4d-48bc-9fb3-f8b9317a8c9f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:31.319380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 624/1254] 10.222.0.50 () {56 vars in 1143 bytes} [Fri Apr 29 18:33:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:31.325300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a757a-d1bc-4194-9a88-13a6c29db9fc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:31.428227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a757a-d1bc-4194-9a88-13a6c29db9fc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:31.461636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7030f95-e966-4475-a892-09dab6885dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:31.462284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030f95-e966-4475-a892-09dab6885dbb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:31.462563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030f95-e966-4475-a892-09dab6885dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:31.462835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030f95-e966-4475-a892-09dab6885dbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:31.533302 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7030f95-e966-4475-a892-09dab6885dbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:31.534778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7030f95-e966-4475-a892-09dab6885dbb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:31.642110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 631/1255] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:33:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:33:31.648293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6698b2-3e4d-48bc-9fb3-f8b9317a8c9f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:31.749245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6698b2-3e4d-48bc-9fb3-f8b9317a8c9f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:31.786233 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9cc06-ec1c-467c-8748-054fb5d2b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:31.787034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9cc06-ec1c-467c-8748-054fb5d2b866 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:31.787388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9cc06-ec1c-467c-8748-054fb5d2b866 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:31.787661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9cc06-ec1c-467c-8748-054fb5d2b866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:31.801297 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9cc06-ec1c-467c-8748-054fb5d2b866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:31.803421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9cc06-ec1c-467c-8748-054fb5d2b866 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2cf9cc06-ec1c-467c-8748-054fb5d2b866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:31.823253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:31.824346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:31.825396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:31.825396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:31.825396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:31.825396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:31.825396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 625/1256] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:31.830469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7030f95-e966-4475-a892-09dab6885dbb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:31.947184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7030f95-e966-4475-a892-09dab6885dbb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:31.989885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef41e8-ba58-430c-823a-05ed28d77ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:31.990621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef41e8-ba58-430c-823a-05ed28d77ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:31.990893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef41e8-ba58-430c-823a-05ed28d77ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:31.991170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef41e8-ba58-430c-823a-05ed28d77ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:31.991886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef41e8-ba58-430c-823a-05ed28d77ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:31.992192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef41e8-ba58-430c-823a-05ed28d77ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:31.993654 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef41e8-ba58-430c-823a-05ed28d77ac2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:32.006975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 632/1257] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:32.013486 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9cc06-ec1c-467c-8748-054fb5d2b866 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:32.129754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9cc06-ec1c-467c-8748-054fb5d2b866 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:32.166420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d0dfd-d90d-4447-aeb2-0f1521a6aca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:32.167212 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d0dfd-d90d-4447-aeb2-0f1521a6aca2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:32.167516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d0dfd-d90d-4447-aeb2-0f1521a6aca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:32.167785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d0dfd-d90d-4447-aeb2-0f1521a6aca2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:32.175478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d0dfd-d90d-4447-aeb2-0f1521a6aca2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:32.177001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d0dfd-d90d-4447-aeb2-0f1521a6aca2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d88d0dfd-d90d-4447-aeb2-0f1521a6aca2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:33:32.192642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:32.193713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:32.194608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:33:32.194608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:33:32.194608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:33:32.194608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:33:32.194608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:33:32.194608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:33:32.194608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:32.194608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:32.194608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 626/1258] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:33:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:32.200225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef41e8-ba58-430c-823a-05ed28d77ac2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:32.306876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef41e8-ba58-430c-823a-05ed28d77ac2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:32.342869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594198e-c02b-4146-b929-189cdd6d2063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:32.343545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594198e-c02b-4146-b929-189cdd6d2063 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:32.343821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594198e-c02b-4146-b929-189cdd6d2063 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:32.344085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594198e-c02b-4146-b929-189cdd6d2063 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:32.344928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594198e-c02b-4146-b929-189cdd6d2063 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:32.345221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594198e-c02b-4146-b929-189cdd6d2063 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:32.346759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594198e-c02b-4146-b929-189cdd6d2063 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:32.359773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 633/1259] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:33:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:32.366849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d0dfd-d90d-4447-aeb2-0f1521a6aca2 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:32.464612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d0dfd-d90d-4447-aeb2-0f1521a6aca2 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:32.499552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4edd0-6500-4bab-9638-deea55752703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:32.500222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4edd0-6500-4bab-9638-deea55752703 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:32.500569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4edd0-6500-4bab-9638-deea55752703 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:32.500796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4edd0-6500-4bab-9638-deea55752703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:32.501623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4edd0-6500-4bab-9638-deea55752703 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=6e934c8e7dd0425cac68eb91e5e2c0ae) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:32.501931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4edd0-6500-4bab-9638-deea55752703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:32.503387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4edd0-6500-4bab-9638-deea55752703 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:32.509593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 627/1260] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:33:32 2022] GET /identity/v3/role_assignments?user.id=6e934c8e7dd0425cac68eb91e5e2c0ae&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=e55ef11844a742f5889243f189bd59bc => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:34.769753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa1a42-d5d6-469c-b4ab-47ff858d5158 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:34.770337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa1a42-d5d6-469c-b4ab-47ff858d5158 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:34.770337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa1a42-d5d6-469c-b4ab-47ff858d5158 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:34.772232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 634/1261] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:34.778701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca59413-406c-4e42-897b-a0d2a92948bb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:34.779408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca59413-406c-4e42-897b-a0d2a92948bb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:34.779584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca59413-406c-4e42-897b-a0d2a92948bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:34.800725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0ca59413-406c-4e42-897b-a0d2a92948bb None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:34.801904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0ca59413-406c-4e42-897b-a0d2a92948bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:34.876721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 628/1262] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:33:34.899432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa1a42-d5d6-469c-b4ab-47ff858d5158 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:35.023333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa1a42-d5d6-469c-b4ab-47ff858d5158 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:35.063465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb469b-1dd7-4d36-8a14-4626132de2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:35.064591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb469b-1dd7-4d36-8a14-4626132de2cf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:35.064990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb469b-1dd7-4d36-8a14-4626132de2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:35.065370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb469b-1dd7-4d36-8a14-4626132de2cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:35.075703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb469b-1dd7-4d36-8a14-4626132de2cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-84fb469b-1dd7-4d36-8a14-4626132de2cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:33:35.083217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:35.084357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:35.085494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 635/1263] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:33:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:35.093400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca59413-406c-4e42-897b-a0d2a92948bb None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:35.196383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca59413-406c-4e42-897b-a0d2a92948bb None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:35.236990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b623cf8-1bae-45b0-952d-22adeeef6577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:35.237658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b623cf8-1bae-45b0-952d-22adeeef6577 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:35.237936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b623cf8-1bae-45b0-952d-22adeeef6577 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:35.238211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b623cf8-1bae-45b0-952d-22adeeef6577 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:35.238952 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b623cf8-1bae-45b0-952d-22adeeef6577 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:35.239258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b623cf8-1bae-45b0-952d-22adeeef6577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:35.240775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b623cf8-1bae-45b0-952d-22adeeef6577 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:35.248910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 629/1264] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:33:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:35.257841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb469b-1dd7-4d36-8a14-4626132de2cf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:35.360801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb469b-1dd7-4d36-8a14-4626132de2cf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:35.393797 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6ea91-6557-41bb-9dec-370ed48fb945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:35.394563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6ea91-6557-41bb-9dec-370ed48fb945 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:35.394834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6ea91-6557-41bb-9dec-370ed48fb945 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:35.395093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6ea91-6557-41bb-9dec-370ed48fb945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:35.405574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e6ea91-6557-41bb-9dec-370ed48fb945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:35.407015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e6ea91-6557-41bb-9dec-370ed48fb945 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c3e6ea91-6557-41bb-9dec-370ed48fb945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:35.423567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:35.424651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:35.425787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:35.425787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:35.425787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:35.425787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:35.425787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 636/1265] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:35.430947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b623cf8-1bae-45b0-952d-22adeeef6577 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:35.529113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b623cf8-1bae-45b0-952d-22adeeef6577 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:35.562069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05cfcd-87fc-4b39-844d-6855cbf06cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:35.562721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05cfcd-87fc-4b39-844d-6855cbf06cfd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:35.562993 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05cfcd-87fc-4b39-844d-6855cbf06cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:35.563261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05cfcd-87fc-4b39-844d-6855cbf06cfd None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:35.563970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05cfcd-87fc-4b39-844d-6855cbf06cfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:35.564268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05cfcd-87fc-4b39-844d-6855cbf06cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:35.565677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05cfcd-87fc-4b39-844d-6855cbf06cfd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:35.578044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 630/1266] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:35.584121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6ea91-6557-41bb-9dec-370ed48fb945 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:35.695904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6ea91-6557-41bb-9dec-370ed48fb945 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:35.730337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df78c8-88d9-46ca-93cf-f95c0c5907d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:35.731127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df78c8-88d9-46ca-93cf-f95c0c5907d1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:35.731401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df78c8-88d9-46ca-93cf-f95c0c5907d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:35.731671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df78c8-88d9-46ca-93cf-f95c0c5907d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:35.739236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df78c8-88d9-46ca-93cf-f95c0c5907d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:35.740853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df78c8-88d9-46ca-93cf-f95c0c5907d1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c5df78c8-88d9-46ca-93cf-f95c0c5907d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:33:35.755037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:35.756141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:35.757443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 637/1267] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:35.763112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05cfcd-87fc-4b39-844d-6855cbf06cfd None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:35.867901 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05cfcd-87fc-4b39-844d-6855cbf06cfd None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:35.904020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00609bbf-953a-41c3-9da1-08293edf76f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:35.904711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00609bbf-953a-41c3-9da1-08293edf76f8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:35.904992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00609bbf-953a-41c3-9da1-08293edf76f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:35.905261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00609bbf-953a-41c3-9da1-08293edf76f8 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:35.906086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00609bbf-953a-41c3-9da1-08293edf76f8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:35.906395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00609bbf-953a-41c3-9da1-08293edf76f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:35.907826 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00609bbf-953a-41c3-9da1-08293edf76f8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:35.924255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 631/1268] 10.222.0.50 () {56 vars in 1143 bytes} [Fri Apr 29 18:33:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:35.931454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df78c8-88d9-46ca-93cf-f95c0c5907d1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:36.050764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df78c8-88d9-46ca-93cf-f95c0c5907d1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:36.083607 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b09fc3-7e0c-4425-b783-357b52ee5604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:36.084582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b09fc3-7e0c-4425-b783-357b52ee5604 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:36.084862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b09fc3-7e0c-4425-b783-357b52ee5604 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:36.085133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b09fc3-7e0c-4425-b783-357b52ee5604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:36.095916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b09fc3-7e0c-4425-b783-357b52ee5604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:36.097406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b09fc3-7e0c-4425-b783-357b52ee5604 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-10b09fc3-7e0c-4425-b783-357b52ee5604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:36.114097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:36.115102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:36.116735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:36.116735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:36.116735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:36.116735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:36.116735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 638/1269] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:36.124014 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00609bbf-953a-41c3-9da1-08293edf76f8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:36.230720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00609bbf-953a-41c3-9da1-08293edf76f8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:36.265469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9542e-d9d1-4b31-884c-80801afbe046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:36.266117 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9542e-d9d1-4b31-884c-80801afbe046 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:36.266397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9542e-d9d1-4b31-884c-80801afbe046 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:36.266661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9542e-d9d1-4b31-884c-80801afbe046 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:36.267371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9542e-d9d1-4b31-884c-80801afbe046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:36.267671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9542e-d9d1-4b31-884c-80801afbe046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:36.269097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9542e-d9d1-4b31-884c-80801afbe046 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:36.280210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 632/1270] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:36.288762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b09fc3-7e0c-4425-b783-357b52ee5604 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:36.389898 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b09fc3-7e0c-4425-b783-357b52ee5604 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:36.429136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62285ae-4bf8-41a1-ad01-04c6955eeeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:36.429921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62285ae-4bf8-41a1-ad01-04c6955eeeee None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:36.430195 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62285ae-4bf8-41a1-ad01-04c6955eeeee None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:36.430459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62285ae-4bf8-41a1-ad01-04c6955eeeee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:36.437488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62285ae-4bf8-41a1-ad01-04c6955eeeee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:36.438928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62285ae-4bf8-41a1-ad01-04c6955eeeee None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b62285ae-4bf8-41a1-ad01-04c6955eeeee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:33:36.453176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:36.454197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:36.459286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:33:36.459286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:33:36.459286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:33:36.459286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:33:36.459286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:33:36.459286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:33:36.459286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:36.459286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:36.459286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 639/1271] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:36.460930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9542e-d9d1-4b31-884c-80801afbe046 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:36.563095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9542e-d9d1-4b31-884c-80801afbe046 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:36.596665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493f823-b709-4b27-ba7c-ebdf95f2cd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:36.597328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493f823-b709-4b27-ba7c-ebdf95f2cd0c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:36.597594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493f823-b709-4b27-ba7c-ebdf95f2cd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:36.597863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493f823-b709-4b27-ba7c-ebdf95f2cd0c None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:36.598633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493f823-b709-4b27-ba7c-ebdf95f2cd0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:36.598933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493f823-b709-4b27-ba7c-ebdf95f2cd0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:36.600519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493f823-b709-4b27-ba7c-ebdf95f2cd0c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:36.613392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 633/1272] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:33:36.619669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62285ae-4bf8-41a1-ad01-04c6955eeeee None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:36.716527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62285ae-4bf8-41a1-ad01-04c6955eeeee None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:36.750350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fe2a8-a7af-40f6-94d3-2fce3327ab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:36.751217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fe2a8-a7af-40f6-94d3-2fce3327ab4d None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:36.751506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fe2a8-a7af-40f6-94d3-2fce3327ab4d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:36.751768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fe2a8-a7af-40f6-94d3-2fce3327ab4d None admin] PATH_INFO: `/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/6e934c8e7dd0425cac68eb91e5e2c0ae/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:36.754000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fe2a8-a7af-40f6-94d3-2fce3327ab4d None admin] RBAC: Authorizing `identity:create_grant(project_id=c7398b586ccc4806bdaa4365ce558959, user_id=6e934c8e7dd0425cac68eb91e5e2c0ae, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:36.755419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fe2a8-a7af-40f6-94d3-2fce3327ab4d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:36.774382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 640/1273] 10.222.0.50 () {58 vars in 1439 bytes} [Fri Apr 29 18:33:36 2022] PUT /identity/v3/projects/c7398b586ccc4806bdaa4365ce558959/users/6e934c8e7dd0425cac68eb91e5e2c0ae/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:33:38.977216 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a0365-12a9-4f68-b7b3-33d23d9a55b6 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:38.978065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a0365-12a9-4f68-b7b3-33d23d9a55b6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:38.978065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a0365-12a9-4f68-b7b3-33d23d9a55b6 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:38.980123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 634/1274] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:38.987569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c968b-7621-4b0f-8580-c237a0e65caf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:38.988062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c968b-7621-4b0f-8580-c237a0e65caf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:38.988062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c968b-7621-4b0f-8580-c237a0e65caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:39.007132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f20c968b-7621-4b0f-8580-c237a0e65caf None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:39.008342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f20c968b-7621-4b0f-8580-c237a0e65caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:39.088132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 641/1275] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Apr 29 18:33:39.113056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a0365-12a9-4f68-b7b3-33d23d9a55b6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:39.216426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a0365-12a9-4f68-b7b3-33d23d9a55b6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:39.250828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846ba88-3a0d-45b4-9ef8-4cf25e10aba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:39.251538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846ba88-3a0d-45b4-9ef8-4cf25e10aba3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:39.251816 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846ba88-3a0d-45b4-9ef8-4cf25e10aba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:39.252084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846ba88-3a0d-45b4-9ef8-4cf25e10aba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:39.262558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846ba88-3a0d-45b4-9ef8-4cf25e10aba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-4846ba88-3a0d-45b4-9ef8-4cf25e10aba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Apr 29 18:33:39.269996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:39.271034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:39.271891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 635/1276] 10.222.0.50 () {56 vars in 1096 bytes} [Fri Apr 29 18:33:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:39.277983 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c968b-7621-4b0f-8580-c237a0e65caf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:39.379311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c968b-7621-4b0f-8580-c237a0e65caf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:39.415388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e7217-3775-4958-9bbd-7cd2051f84a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:39.416046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e7217-3775-4958-9bbd-7cd2051f84a4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:39.416330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e7217-3775-4958-9bbd-7cd2051f84a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:39.416628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e7217-3775-4958-9bbd-7cd2051f84a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:39.417369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e7217-3775-4958-9bbd-7cd2051f84a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:39.417675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e7217-3775-4958-9bbd-7cd2051f84a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:39.419054 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e7217-3775-4958-9bbd-7cd2051f84a4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:39.427885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 642/1277] 10.222.0.50 () {56 vars in 1099 bytes} [Fri Apr 29 18:33:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:39.434394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846ba88-3a0d-45b4-9ef8-4cf25e10aba3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:39.543264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846ba88-3a0d-45b4-9ef8-4cf25e10aba3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:39.576343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63207f18-7d92-4d4a-8595-29d8217c2225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:39.577158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63207f18-7d92-4d4a-8595-29d8217c2225 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:39.577435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63207f18-7d92-4d4a-8595-29d8217c2225 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:39.577704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63207f18-7d92-4d4a-8595-29d8217c2225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:39.588012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63207f18-7d92-4d4a-8595-29d8217c2225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:39.589517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63207f18-7d92-4d4a-8595-29d8217c2225 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-63207f18-7d92-4d4a-8595-29d8217c2225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:39.607159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:39.608255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:39.609345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:39.609345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:39.609345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:39.609345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:39.609345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 636/1278] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:39.614713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e7217-3775-4958-9bbd-7cd2051f84a4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:39.727959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e7217-3775-4958-9bbd-7cd2051f84a4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:39.761058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47d444-bdb5-42b3-83f4-670bd42c7655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:39.761701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47d444-bdb5-42b3-83f4-670bd42c7655 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:39.761975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47d444-bdb5-42b3-83f4-670bd42c7655 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:39.762241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47d444-bdb5-42b3-83f4-670bd42c7655 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:39.762953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47d444-bdb5-42b3-83f4-670bd42c7655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:39.763248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47d444-bdb5-42b3-83f4-670bd42c7655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:39.764644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47d444-bdb5-42b3-83f4-670bd42c7655 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:39.776025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 643/1279] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:39.785199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63207f18-7d92-4d4a-8595-29d8217c2225 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:39.885653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63207f18-7d92-4d4a-8595-29d8217c2225 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:39.920515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937ccc1-d522-47db-af41-4c20cfca7309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:39.921318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937ccc1-d522-47db-af41-4c20cfca7309 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:39.921599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937ccc1-d522-47db-af41-4c20cfca7309 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:39.921863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937ccc1-d522-47db-af41-4c20cfca7309 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:39.929078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937ccc1-d522-47db-af41-4c20cfca7309 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:39.930527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937ccc1-d522-47db-af41-4c20cfca7309 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0937ccc1-d522-47db-af41-4c20cfca7309 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Apr 29 18:33:39.944960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:39.946181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:39.947108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 637/1280] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:39.952691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47d444-bdb5-42b3-83f4-670bd42c7655 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:40.057255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47d444-bdb5-42b3-83f4-670bd42c7655 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:40.091709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800cfcd-1ca4-4555-ab6e-3a5a2e5df08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:40.092368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800cfcd-1ca4-4555-ab6e-3a5a2e5df08b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:40.092674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800cfcd-1ca4-4555-ab6e-3a5a2e5df08b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:40.092941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800cfcd-1ca4-4555-ab6e-3a5a2e5df08b None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:40.093765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800cfcd-1ca4-4555-ab6e-3a5a2e5df08b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:40.094150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800cfcd-1ca4-4555-ab6e-3a5a2e5df08b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:40.095573 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800cfcd-1ca4-4555-ab6e-3a5a2e5df08b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:40.111715 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 644/1281] 10.222.0.50 () {56 vars in 1143 bytes} [Fri Apr 29 18:33:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:40.117799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937ccc1-d522-47db-af41-4c20cfca7309 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:40.213486 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937ccc1-d522-47db-af41-4c20cfca7309 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:40.253862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5e802-c583-4189-a191-dbe8846fb5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:40.254507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5e802-c583-4189-a191-dbe8846fb5db None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:40.254784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5e802-c583-4189-a191-dbe8846fb5db None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:40.255051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5e802-c583-4189-a191-dbe8846fb5db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:40.324392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5e802-c583-4189-a191-dbe8846fb5db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:40.325869 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5e802-c583-4189-a191-dbe8846fb5db None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:40.423488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 638/1282] 10.222.0.50 () {58 vars in 1282 bytes} [Fri Apr 29 18:33:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Apr 29 18:33:40.430793 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800cfcd-1ca4-4555-ab6e-3a5a2e5df08b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:40.538455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800cfcd-1ca4-4555-ab6e-3a5a2e5df08b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:40.571551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854233ca-f6c8-4911-a682-1a1f296ecf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:40.572323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854233ca-f6c8-4911-a682-1a1f296ecf35 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:40.572632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854233ca-f6c8-4911-a682-1a1f296ecf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:40.572896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854233ca-f6c8-4911-a682-1a1f296ecf35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:40.582719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854233ca-f6c8-4911-a682-1a1f296ecf35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:40.584175 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854233ca-f6c8-4911-a682-1a1f296ecf35 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-854233ca-f6c8-4911-a682-1a1f296ecf35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Apr 29 18:33:40.601583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:40.602531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:40.603518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Apr 29 18:33:40.603518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Apr 29 18:33:40.603518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Apr 29 18:33:40.603518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:40.603518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 645/1283] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:40.608856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5e802-c583-4189-a191-dbe8846fb5db None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:40.714192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5e802-c583-4189-a191-dbe8846fb5db None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:40.748516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08aadc5-8de5-4f0d-b5e0-4cbde98f3365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:40.749190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08aadc5-8de5-4f0d-b5e0-4cbde98f3365 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:40.749468 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08aadc5-8de5-4f0d-b5e0-4cbde98f3365 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:40.749742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08aadc5-8de5-4f0d-b5e0-4cbde98f3365 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:40.750446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08aadc5-8de5-4f0d-b5e0-4cbde98f3365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:40.750748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08aadc5-8de5-4f0d-b5e0-4cbde98f3365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:40.752134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08aadc5-8de5-4f0d-b5e0-4cbde98f3365 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:40.763222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 639/1284] 10.222.0.50 () {56 vars in 1109 bytes} [Fri Apr 29 18:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:40.769309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854233ca-f6c8-4911-a682-1a1f296ecf35 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:40.871231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854233ca-f6c8-4911-a682-1a1f296ecf35 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:40.910273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3f368-ca1c-4dae-ad23-0cb61410c84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:40.911058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3f368-ca1c-4dae-ad23-0cb61410c84a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:40.911345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3f368-ca1c-4dae-ad23-0cb61410c84a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:40.911614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3f368-ca1c-4dae-ad23-0cb61410c84a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:40.918806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3f368-ca1c-4dae-ad23-0cb61410c84a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:40.920257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3f368-ca1c-4dae-ad23-0cb61410c84a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0ca3f368-ca1c-4dae-ad23-0cb61410c84a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Apr 29 18:33:40.933756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:40.934764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:40.936199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Apr 29 18:33:40.936199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Apr 29 18:33:40.936199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Apr 29 18:33:40.936199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Apr 29 18:33:40.936199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Apr 29 18:33:40.936199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Apr 29 18:33:40.936199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Apr 29 18:33:40.936199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Apr 29 18:33:40.936199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 646/1285] 10.222.0.50 () {56 vars in 1111 bytes} [Fri Apr 29 18:33:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:40.943954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08aadc5-8de5-4f0d-b5e0-4cbde98f3365 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:41.056713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08aadc5-8de5-4f0d-b5e0-4cbde98f3365 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:41.093446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef74bf-9f09-4cae-8948-0a5a1c29a1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:41.094478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef74bf-9f09-4cae-8948-0a5a1c29a1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:41.094943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef74bf-9f09-4cae-8948-0a5a1c29a1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:41.095400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef74bf-9f09-4cae-8948-0a5a1c29a1b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:41.096718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef74bf-9f09-4cae-8948-0a5a1c29a1b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:41.097227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef74bf-9f09-4cae-8948-0a5a1c29a1b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:41.099597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef74bf-9f09-4cae-8948-0a5a1c29a1b3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:41.118605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 640/1286] 10.222.0.50 () {56 vars in 1148 bytes} [Fri Apr 29 18:33:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:41.125199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3f368-ca1c-4dae-ad23-0cb61410c84a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:41.226059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3f368-ca1c-4dae-ad23-0cb61410c84a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:41.260658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c6edc-1da9-48ab-b089-12f20fd04d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:41.261312 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c6edc-1da9-48ab-b089-12f20fd04d85 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:41.261589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c6edc-1da9-48ab-b089-12f20fd04d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:41.261860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c6edc-1da9-48ab-b089-12f20fd04d85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:41.262678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c6edc-1da9-48ab-b089-12f20fd04d85 None admin] RBAC: Adding query filter params (role.id=e55ef11844a742f5889243f189bd59bc, scope.project.id=c7398b586ccc4806bdaa4365ce558959, user.id=6e934c8e7dd0425cac68eb91e5e2c0ae) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:41.262987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c6edc-1da9-48ab-b089-12f20fd04d85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:41.264429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c6edc-1da9-48ab-b089-12f20fd04d85 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:41.270334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 647/1287] 10.222.0.50 () {56 vars in 1374 bytes} [Fri Apr 29 18:33:41 2022] GET /identity/v3/role_assignments?user.id=6e934c8e7dd0425cac68eb91e5e2c0ae&scope.project.id=c7398b586ccc4806bdaa4365ce558959&role.id=e55ef11844a742f5889243f189bd59bc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:43.523825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53384a50-63a6-4216-b730-3ee3e4fb76de None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:43.524766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53384a50-63a6-4216-b730-3ee3e4fb76de None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:43.524766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53384a50-63a6-4216-b730-3ee3e4fb76de None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:43.526521 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 641/1288] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:43.532103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512a535-6d19-4ccc-851c-bf32fe8cbb19 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:43.532550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512a535-6d19-4ccc-851c-bf32fe8cbb19 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:43.532642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512a535-6d19-4ccc-851c-bf32fe8cbb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:43.551670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9512a535-6d19-4ccc-851c-bf32fe8cbb19 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:43.552881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9512a535-6d19-4ccc-851c-bf32fe8cbb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:43.619848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 648/1289] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:33:43.642346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53384a50-63a6-4216-b730-3ee3e4fb76de None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:43.750288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53384a50-63a6-4216-b730-3ee3e4fb76de None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:43.783419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affc9d2-5fea-45f5-b041-a88713227c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:43.784126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affc9d2-5fea-45f5-b041-a88713227c1e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:43.784407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affc9d2-5fea-45f5-b041-a88713227c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:43.784691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affc9d2-5fea-45f5-b041-a88713227c1e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:43.785327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affc9d2-5fea-45f5-b041-a88713227c1e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:43.786684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affc9d2-5fea-45f5-b041-a88713227c1e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-9affc9d2-5fea-45f5-b041-a88713227c1e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:43.798226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:43.799374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:43.800328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 642/1290] 10.222.0.50 () {56 vars in 1117 bytes} [Fri Apr 29 18:33:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:43.806490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512a535-6d19-4ccc-851c-bf32fe8cbb19 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:43.910891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512a535-6d19-4ccc-851c-bf32fe8cbb19 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:43.946075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4501a15-a9e5-4cda-b2c1-09289ea96865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:43.946759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4501a15-a9e5-4cda-b2c1-09289ea96865 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:43.947039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4501a15-a9e5-4cda-b2c1-09289ea96865 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:43.947293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4501a15-a9e5-4cda-b2c1-09289ea96865 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:43.947991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4501a15-a9e5-4cda-b2c1-09289ea96865 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:43.948293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4501a15-a9e5-4cda-b2c1-09289ea96865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:43.949700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4501a15-a9e5-4cda-b2c1-09289ea96865 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:43.954614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 649/1291] 10.222.0.50 () {56 vars in 1116 bytes} [Fri Apr 29 18:33:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:43.961720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affc9d2-5fea-45f5-b041-a88713227c1e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:44.071142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affc9d2-5fea-45f5-b041-a88713227c1e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:44.104335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f040c3-2dd6-4e9a-a840-711d7812f188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:44.105013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f040c3-2dd6-4e9a-a840-711d7812f188 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:44.105286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f040c3-2dd6-4e9a-a840-711d7812f188 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:44.105550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f040c3-2dd6-4e9a-a840-711d7812f188 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:44.106238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f040c3-2dd6-4e9a-a840-711d7812f188 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:44.106536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f040c3-2dd6-4e9a-a840-711d7812f188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:44.107896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f040c3-2dd6-4e9a-a840-711d7812f188 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:44.112313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 643/1292] 10.222.0.50 () {56 vars in 1116 bytes} [Fri Apr 29 18:33:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:33:46.312528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e390c4-dcbf-4388-83f8-03765d9fc9d8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:46.313092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e390c4-dcbf-4388-83f8-03765d9fc9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:46.313092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e390c4-dcbf-4388-83f8-03765d9fc9d8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:46.314461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 650/1293] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:46.320830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbdd6d-c1d9-4970-9a70-2577d70085ec None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:46.321554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbdd6d-c1d9-4970-9a70-2577d70085ec None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:46.321738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbdd6d-c1d9-4970-9a70-2577d70085ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:46.343850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-79fbdd6d-c1d9-4970-9a70-2577d70085ec None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:46.345055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-79fbdd6d-c1d9-4970-9a70-2577d70085ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:46.415613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 644/1294] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:33:46.438853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e390c4-dcbf-4388-83f8-03765d9fc9d8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:46.552007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e390c4-dcbf-4388-83f8-03765d9fc9d8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:46.586488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c85052-3f3b-4c80-b3a1-ac02dc372347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:46.587067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c85052-3f3b-4c80-b3a1-ac02dc372347 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:46.587342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c85052-3f3b-4c80-b3a1-ac02dc372347 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:46.587948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c85052-3f3b-4c80-b3a1-ac02dc372347 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:46.588689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c85052-3f3b-4c80-b3a1-ac02dc372347 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:46.590067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c85052-3f3b-4c80-b3a1-ac02dc372347 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:46.606813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 651/1295] 10.222.0.50 () {60 vars in 1141 bytes} [Fri Apr 29 18:33:46 2022] POST /identity/v3/services => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:33:48.865468 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c7d6d-d0d7-410c-8796-fbd9f324ba0f None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:48.866394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c7d6d-d0d7-410c-8796-fbd9f324ba0f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:48.866394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c7d6d-d0d7-410c-8796-fbd9f324ba0f None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:48.868454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 645/1296] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:48.875292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6161fec-2676-4679-b37b-455a3e86aee1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:48.876025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6161fec-2676-4679-b37b-455a3e86aee1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:48.876025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6161fec-2676-4679-b37b-455a3e86aee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:48.895441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a6161fec-2676-4679-b37b-455a3e86aee1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:48.896668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a6161fec-2676-4679-b37b-455a3e86aee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:48.974114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 652/1297] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:33:49.001275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c7d6d-d0d7-410c-8796-fbd9f324ba0f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:49.118413 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c7d6d-d0d7-410c-8796-fbd9f324ba0f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:49.151248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42958e-7fde-4114-933c-6f345dc26b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:49.151981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42958e-7fde-4114-933c-6f345dc26b55 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:49.152263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42958e-7fde-4114-933c-6f345dc26b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:49.152537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42958e-7fde-4114-933c-6f345dc26b55 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:49.153206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42958e-7fde-4114-933c-6f345dc26b55 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:49.154596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42958e-7fde-4114-933c-6f345dc26b55 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ee42958e-7fde-4114-933c-6f345dc26b55 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:49.169613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:49.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:49.171799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 646/1298] 10.222.0.50 () {56 vars in 1117 bytes} [Fri Apr 29 18:33:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:49.177226 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6161fec-2676-4679-b37b-455a3e86aee1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:49.280575 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6161fec-2676-4679-b37b-455a3e86aee1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:49.315295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d27ea7-e44f-485d-88f8-023f63b674a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:49.315945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d27ea7-e44f-485d-88f8-023f63b674a6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:49.316226 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d27ea7-e44f-485d-88f8-023f63b674a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:49.316536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d27ea7-e44f-485d-88f8-023f63b674a6 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:49.317235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d27ea7-e44f-485d-88f8-023f63b674a6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:49.317543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d27ea7-e44f-485d-88f8-023f63b674a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:49.318942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d27ea7-e44f-485d-88f8-023f63b674a6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:49.324159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 653/1299] 10.222.0.50 () {56 vars in 1116 bytes} [Fri Apr 29 18:33:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:49.333610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42958e-7fde-4114-933c-6f345dc26b55 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:49.433301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42958e-7fde-4114-933c-6f345dc26b55 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:49.468049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8853c4-02c0-4e6b-8e50-6a3035a7950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:49.468717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8853c4-02c0-4e6b-8e50-6a3035a7950f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:49.468994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8853c4-02c0-4e6b-8e50-6a3035a7950f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:49.469261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8853c4-02c0-4e6b-8e50-6a3035a7950f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:49.470030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8853c4-02c0-4e6b-8e50-6a3035a7950f None admin] RBAC: Adding query filter params (interface=public, service_id=0f3d73e1ce1d48768036d3d61e38bb91, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:49.470332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8853c4-02c0-4e6b-8e50-6a3035a7950f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:49.471681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8853c4-02c0-4e6b-8e50-6a3035a7950f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:49.477130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 647/1300] 10.222.0.50 () {56 vars in 1251 bytes} [Fri Apr 29 18:33:49 2022] GET /identity/v3/endpoints?service_id=0f3d73e1ce1d48768036d3d61e38bb91&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:49.485514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d27ea7-e44f-485d-88f8-023f63b674a6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:49.581347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d27ea7-e44f-485d-88f8-023f63b674a6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:49.615771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49f7db-d1f4-4f21-aebd-2de66e0033b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:49.616420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f7db-d1f4-4f21-aebd-2de66e0033b5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:49.616721 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f7db-d1f4-4f21-aebd-2de66e0033b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:49.616986 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f7db-d1f4-4f21-aebd-2de66e0033b5 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:49.617662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49f7db-d1f4-4f21-aebd-2de66e0033b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:49.619031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49f7db-d1f4-4f21-aebd-2de66e0033b5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:49.628274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 654/1301] 10.222.0.50 () {56 vars in 1087 bytes} [Fri Apr 29 18:33:49 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:33:51.844738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d98376-d8ae-4331-b0ff-6cd4b03d3450 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:51.845299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d98376-d8ae-4331-b0ff-6cd4b03d3450 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:51.845299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d98376-d8ae-4331-b0ff-6cd4b03d3450 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:51.846632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 648/1302] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:33:51.853147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f41fe-1640-48e2-a072-0f236f522b31 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:51.853819 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f41fe-1640-48e2-a072-0f236f522b31 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:51.854009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f41fe-1640-48e2-a072-0f236f522b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:51.882507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-da6f41fe-1640-48e2-a072-0f236f522b31 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:33:51.883645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-da6f41fe-1640-48e2-a072-0f236f522b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:33:51.952973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 655/1303] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:33:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Apr 29 18:33:51.978804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d98376-d8ae-4331-b0ff-6cd4b03d3450 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:52.091436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d98376-d8ae-4331-b0ff-6cd4b03d3450 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:52.127232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a18d1-928d-490f-9801-777bdd743a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:52.128013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a18d1-928d-490f-9801-777bdd743a9e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:52.128286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a18d1-928d-490f-9801-777bdd743a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:52.128587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a18d1-928d-490f-9801-777bdd743a9e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:52.129217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a18d1-928d-490f-9801-777bdd743a9e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:52.130753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a18d1-928d-490f-9801-777bdd743a9e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-510a18d1-928d-490f-9801-777bdd743a9e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Apr 29 18:33:52.142593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Apr 29 18:33:52.143704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Apr 29 18:33:52.144628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 649/1304] 10.222.0.50 () {56 vars in 1117 bytes} [Fri Apr 29 18:33:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Apr 29 18:33:52.151365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f41fe-1640-48e2-a072-0f236f522b31 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:52.259585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f41fe-1640-48e2-a072-0f236f522b31 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:52.297070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3f503-67a0-4b77-951e-5344c7b50012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:52.297714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f503-67a0-4b77-951e-5344c7b50012 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:52.298022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f503-67a0-4b77-951e-5344c7b50012 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:52.298284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f503-67a0-4b77-951e-5344c7b50012 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:52.298985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3f503-67a0-4b77-951e-5344c7b50012 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:33:52.299284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3f503-67a0-4b77-951e-5344c7b50012 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:52.300673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3f503-67a0-4b77-951e-5344c7b50012 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:52.306440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 656/1305] 10.222.0.50 () {56 vars in 1116 bytes} [Fri Apr 29 18:33:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:33:52.315639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a18d1-928d-490f-9801-777bdd743a9e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:33:52.418836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a18d1-928d-490f-9801-777bdd743a9e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:33:52.451945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba46d7-ba90-40cb-8165-7701bfb177cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:33:52.452600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba46d7-ba90-40cb-8165-7701bfb177cc None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:33:52.452895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba46d7-ba90-40cb-8165-7701bfb177cc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:33:52.453160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba46d7-ba90-40cb-8165-7701bfb177cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:33:52.453882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba46d7-ba90-40cb-8165-7701bfb177cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:33:52.455246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba46d7-ba90-40cb-8165-7701bfb177cc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:33:52.476408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 650/1306] 10.222.0.50 () {60 vars in 1144 bytes} [Fri Apr 29 18:33:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:34:55.801778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc555-95a0-4e55-bdf1-858912debc0a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:34:55.802425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc555-95a0-4e55-bdf1-858912debc0a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:34:55.802425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc555-95a0-4e55-bdf1-858912debc0a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:34:55.803663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 657/1307] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:34:55.808930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271db474-579a-4fd2-9e4f-9e37adbe1b6a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:34:55.809411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271db474-579a-4fd2-9e4f-9e37adbe1b6a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:34:55.809522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271db474-579a-4fd2-9e4f-9e37adbe1b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:34:55.889334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-271db474-579a-4fd2-9e4f-9e37adbe1b6a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:34:55.890534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-271db474-579a-4fd2-9e4f-9e37adbe1b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:34:56.017239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 651/1308] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:34:56.082966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308ad4-6694-447d-9240-34b53125316f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:34:56.083432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308ad4-6694-447d-9240-34b53125316f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:34:56.083863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308ad4-6694-447d-9240-34b53125316f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:34:56.086612 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 658/1309] 10.222.0.50 () {54 vars in 964 bytes} [Fri Apr 29 18:34:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:34:56.093508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40222dff-53bc-4676-9a4a-ddad2507221d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:34:56.093799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40222dff-53bc-4676-9a4a-ddad2507221d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:34:56.094076 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40222dff-53bc-4676-9a4a-ddad2507221d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:34:56.221603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-40222dff-53bc-4676-9a4a-ddad2507221d None None] MFA Rules not processed for user `07849b2925334763b119111ed52dd34c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:34:56.223911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-40222dff-53bc-4676-9a4a-ddad2507221d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:34:56.313747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 652/1310] 10.222.0.50 () {58 vars in 1061 bytes} [Fri Apr 29 18:34:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:34:56.323177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a308ad4-6694-447d-9240-34b53125316f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:34:56.325158 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3a308ad4-6694-447d-9240-34b53125316f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:34:56.440195 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a308ad4-6694-447d-9240-34b53125316f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:34:56.515311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2217fa-3596-4d53-b117-5ba46882b27b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:34:56.516266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2217fa-3596-4d53-b117-5ba46882b27b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:34:56.516674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2217fa-3596-4d53-b117-5ba46882b27b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:34:56.517047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2217fa-3596-4d53-b117-5ba46882b27b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:34:56.518803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ae2217fa-3596-4d53-b117-5ba46882b27b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:34:56.631076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2217fa-3596-4d53-b117-5ba46882b27b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:34:56.632658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2217fa-3596-4d53-b117-5ba46882b27b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:34:56.653542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 659/1311] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:34:59.211817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7d163-400f-4af7-8657-328d9c0499a9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:34:59.212353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7d163-400f-4af7-8657-328d9c0499a9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:34:59.212353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7d163-400f-4af7-8657-328d9c0499a9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:34:59.213736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 653/1312] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:34:59.219206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18febce-d5ab-4201-b5f6-c8422acb88a3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:34:59.219652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18febce-d5ab-4201-b5f6-c8422acb88a3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:34:59.219765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18febce-d5ab-4201-b5f6-c8422acb88a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:34:59.310389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e18febce-d5ab-4201-b5f6-c8422acb88a3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:34:59.311699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e18febce-d5ab-4201-b5f6-c8422acb88a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:34:59.411975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 660/1313] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:34:59.462761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821adc9-142d-4edb-9c96-0cc5a9984f45 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:34:59.463147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821adc9-142d-4edb-9c96-0cc5a9984f45 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:34:59.463507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821adc9-142d-4edb-9c96-0cc5a9984f45 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:34:59.464938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 654/1314] 10.222.0.50 () {54 vars in 964 bytes} [Fri Apr 29 18:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:34:59.470869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35731264-185c-4ab8-92cc-45e9c5586791 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:34:59.471159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35731264-185c-4ab8-92cc-45e9c5586791 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:34:59.471449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35731264-185c-4ab8-92cc-45e9c5586791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:34:59.543556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-35731264-185c-4ab8-92cc-45e9c5586791 None None] MFA Rules not processed for user `07849b2925334763b119111ed52dd34c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:34:59.545055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-35731264-185c-4ab8-92cc-45e9c5586791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:34:59.621842 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 661/1315] 10.222.0.50 () {58 vars in 1061 bytes} [Fri Apr 29 18:34:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:34:59.631093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821adc9-142d-4edb-9c96-0cc5a9984f45 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:34:59.632652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7821adc9-142d-4edb-9c96-0cc5a9984f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:34:59.736529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821adc9-142d-4edb-9c96-0cc5a9984f45 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:34:59.814377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa4c51-53d9-48a7-acca-4b14ee821d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:34:59.815045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa4c51-53d9-48a7-acca-4b14ee821d48 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:34:59.815316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa4c51-53d9-48a7-acca-4b14ee821d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:34:59.815584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa4c51-53d9-48a7-acca-4b14ee821d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:34:59.817272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-33fa4c51-53d9-48a7-acca-4b14ee821d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:34:59.924198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa4c51-53d9-48a7-acca-4b14ee821d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:34:59.927857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa4c51-53d9-48a7-acca-4b14ee821d48 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:34:59.945293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 655/1316] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:34:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:02.801015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecaf81-fc92-4c3b-b422-e62270272aab None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:02.801616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecaf81-fc92-4c3b-b422-e62270272aab None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:02.801616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecaf81-fc92-4c3b-b422-e62270272aab None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:02.803152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 662/1317] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:02.808403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060cd26-3c7d-4b62-bb7d-24f2fb23c00c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:02.808907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060cd26-3c7d-4b62-bb7d-24f2fb23c00c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:02.809011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060cd26-3c7d-4b62-bb7d-24f2fb23c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:02.876412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7060cd26-3c7d-4b62-bb7d-24f2fb23c00c None None] MFA Rules not processed for user `c920286345e24cc09a50a6ef9210d7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:02.877546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7060cd26-3c7d-4b62-bb7d-24f2fb23c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:02.972078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 656/1318] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:03.013385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecaf81-fc92-4c3b-b422-e62270272aab None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:03.014974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-09ecaf81-fc92-4c3b-b422-e62270272aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:03.126982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecaf81-fc92-4c3b-b422-e62270272aab None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:03.203028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857f4af-c4d4-43af-8659-d46e89c8dcaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:03.203687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857f4af-c4d4-43af-8659-d46e89c8dcaa service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:03.203970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857f4af-c4d4-43af-8659-d46e89c8dcaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:03.204243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857f4af-c4d4-43af-8659-d46e89c8dcaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:03.205925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a857f4af-c4d4-43af-8659-d46e89c8dcaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:03.317383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857f4af-c4d4-43af-8659-d46e89c8dcaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:03.318839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857f4af-c4d4-43af-8659-d46e89c8dcaa service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:03.337598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 663/1319] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:03 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:06.092442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b4554-a3f4-4e86-8286-47f71a99a1aa None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:06.093396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b4554-a3f4-4e86-8286-47f71a99a1aa None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:06.093396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b4554-a3f4-4e86-8286-47f71a99a1aa None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:06.095602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 657/1320] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:06.104974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1369e-083c-4e5c-acf9-b87838149b6e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:06.105691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1369e-083c-4e5c-acf9-b87838149b6e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:06.105879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1369e-083c-4e5c-acf9-b87838149b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:06.178150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fca1369e-083c-4e5c-acf9-b87838149b6e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:06.179326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fca1369e-083c-4e5c-acf9-b87838149b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:06.290885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 664/1321] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:06.329196 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b4554-a3f4-4e86-8286-47f71a99a1aa None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:06.330761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-517b4554-a3f4-4e86-8286-47f71a99a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:06.435577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b4554-a3f4-4e86-8286-47f71a99a1aa None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:06.507155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef46c37-9d37-472e-a040-6cd2850e96ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:06.507790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef46c37-9d37-472e-a040-6cd2850e96ce service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:06.508070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef46c37-9d37-472e-a040-6cd2850e96ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:06.508339 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef46c37-9d37-472e-a040-6cd2850e96ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:06.510002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bef46c37-9d37-472e-a040-6cd2850e96ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:06.617514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef46c37-9d37-472e-a040-6cd2850e96ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:06.618962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef46c37-9d37-472e-a040-6cd2850e96ce service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:06.637104 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 658/1322] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:09.752543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b8bd3-6faf-4e65-bcef-e568ce5a12d8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:09.753167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b8bd3-6faf-4e65-bcef-e568ce5a12d8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:09.753167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b8bd3-6faf-4e65-bcef-e568ce5a12d8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:09.754789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 665/1323] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:09.759811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2df37f-3506-43d4-96a6-4748a7f89ed1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:09.760232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2df37f-3506-43d4-96a6-4748a7f89ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:09.760401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2df37f-3506-43d4-96a6-4748a7f89ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:09.829194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3e2df37f-3506-43d4-96a6-4748a7f89ed1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:09.830380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3e2df37f-3506-43d4-96a6-4748a7f89ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:09.926291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 659/1324] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:09.963761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b8bd3-6faf-4e65-bcef-e568ce5a12d8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:09.965340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-113b8bd3-6faf-4e65-bcef-e568ce5a12d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:10.071904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b8bd3-6faf-4e65-bcef-e568ce5a12d8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:10.145251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554ee02-fc9e-4252-b8a4-ce60d83c0b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:10.145905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554ee02-fc9e-4252-b8a4-ce60d83c0b3d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:10.146183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554ee02-fc9e-4252-b8a4-ce60d83c0b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:10.146448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554ee02-fc9e-4252-b8a4-ce60d83c0b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:10.148108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c554ee02-fc9e-4252-b8a4-ce60d83c0b3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:10.262582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554ee02-fc9e-4252-b8a4-ce60d83c0b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:10.263999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554ee02-fc9e-4252-b8a4-ce60d83c0b3d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:10.283021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 666/1325] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:12.601412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb6774-4358-4a87-9b2e-229cae33af7b None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:12.601960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb6774-4358-4a87-9b2e-229cae33af7b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:12.601960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb6774-4358-4a87-9b2e-229cae33af7b None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:12.603270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 660/1326] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:12.608822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97159b6-f8d8-4366-9ac9-9586ff5b187e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:12.609235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97159b6-f8d8-4366-9ac9-9586ff5b187e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:12.609371 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97159b6-f8d8-4366-9ac9-9586ff5b187e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:12.672222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d97159b6-f8d8-4366-9ac9-9586ff5b187e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:12.673421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d97159b6-f8d8-4366-9ac9-9586ff5b187e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:12.769178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 667/1327] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:12.805144 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb6774-4358-4a87-9b2e-229cae33af7b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:12.806714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9fbb6774-4358-4a87-9b2e-229cae33af7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:12.910291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb6774-4358-4a87-9b2e-229cae33af7b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:12.984299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06485931-41b1-4ba0-9154-98e05f2d2ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:12.984972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485931-41b1-4ba0-9154-98e05f2d2ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:12.985276 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485931-41b1-4ba0-9154-98e05f2d2ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:12.985575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485931-41b1-4ba0-9154-98e05f2d2ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:12.987142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-06485931-41b1-4ba0-9154-98e05f2d2ad4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:13.094265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06485931-41b1-4ba0-9154-98e05f2d2ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:13.095700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06485931-41b1-4ba0-9154-98e05f2d2ad4 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:13.113936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 661/1328] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:15.452933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bdac8-cf40-4092-949b-3bf50ee945ab None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:15.476848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bdac8-cf40-4092-949b-3bf50ee945ab None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:15.476848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bdac8-cf40-4092-949b-3bf50ee945ab None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:15.476848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 668/1329] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:15.477229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12de56-4738-48a7-a422-d184faa31a6c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:15.477229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12de56-4738-48a7-a422-d184faa31a6c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:15.477229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12de56-4738-48a7-a422-d184faa31a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:15.534399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-fe12de56-4738-48a7-a422-d184faa31a6c None None] MFA Rules not processed for user `c920286345e24cc09a50a6ef9210d7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:15.535893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fe12de56-4738-48a7-a422-d184faa31a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:15.636760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 662/1330] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:15.673778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bdac8-cf40-4092-949b-3bf50ee945ab None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:15.676235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f54bdac8-cf40-4092-949b-3bf50ee945ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:15.787289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bdac8-cf40-4092-949b-3bf50ee945ab None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:15.860266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bad5eb-1bee-4a4b-ac80-4c9809ed9afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:15.860971 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bad5eb-1bee-4a4b-ac80-4c9809ed9afb service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:15.861251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bad5eb-1bee-4a4b-ac80-4c9809ed9afb service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:15.861745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bad5eb-1bee-4a4b-ac80-4c9809ed9afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:15.863321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e2bad5eb-1bee-4a4b-ac80-4c9809ed9afb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:15.970591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bad5eb-1bee-4a4b-ac80-4c9809ed9afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:15.972029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bad5eb-1bee-4a4b-ac80-4c9809ed9afb service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:15.989774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 669/1331] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:15 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:18.498405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2f093-f48c-40c4-8b6a-7658e9375166 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:18.498953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2f093-f48c-40c4-8b6a-7658e9375166 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:18.498953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2f093-f48c-40c4-8b6a-7658e9375166 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:18.500236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 663/1332] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:18.505631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed11003-6b5b-49f1-a208-30d89446d2dc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:18.506095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed11003-6b5b-49f1-a208-30d89446d2dc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:18.506204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed11003-6b5b-49f1-a208-30d89446d2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:18.579484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1ed11003-6b5b-49f1-a208-30d89446d2dc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:18.580657 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1ed11003-6b5b-49f1-a208-30d89446d2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:18.680680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 670/1333] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:18.717872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2f093-f48c-40c4-8b6a-7658e9375166 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:18.720176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8fe2f093-f48c-40c4-8b6a-7658e9375166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:18.836699 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2f093-f48c-40c4-8b6a-7658e9375166 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:18.914337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfe4fc-f8a4-405a-a7cc-6b80c24e5453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:18.914966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfe4fc-f8a4-405a-a7cc-6b80c24e5453 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:18.915242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfe4fc-f8a4-405a-a7cc-6b80c24e5453 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:18.915533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfe4fc-f8a4-405a-a7cc-6b80c24e5453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:18.917235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-54cfe4fc-f8a4-405a-a7cc-6b80c24e5453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:19.027746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfe4fc-f8a4-405a-a7cc-6b80c24e5453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:19.029308 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfe4fc-f8a4-405a-a7cc-6b80c24e5453 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:19.047911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 664/1334] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:21.348761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0690fe5-be9d-424f-b603-c233bdd62b13 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:21.349339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0690fe5-be9d-424f-b603-c233bdd62b13 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:21.349339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0690fe5-be9d-424f-b603-c233bdd62b13 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:21.351316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 671/1335] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:21.359954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46273506-dc4f-49dd-a3fb-30e20c0baa91 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:21.360687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46273506-dc4f-49dd-a3fb-30e20c0baa91 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:21.360812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46273506-dc4f-49dd-a3fb-30e20c0baa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:21.442740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-46273506-dc4f-49dd-a3fb-30e20c0baa91 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:21.443974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-46273506-dc4f-49dd-a3fb-30e20c0baa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:21.544942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 665/1336] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:21.593031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0690fe5-be9d-424f-b603-c233bdd62b13 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:21.594571 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b0690fe5-be9d-424f-b603-c233bdd62b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:21.709678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0690fe5-be9d-424f-b603-c233bdd62b13 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:21.783314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e2ff3-cda2-42d8-aa54-d0935f883efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:21.783966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e2ff3-cda2-42d8-aa54-d0935f883efd service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:21.784242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e2ff3-cda2-42d8-aa54-d0935f883efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:21.784525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e2ff3-cda2-42d8-aa54-d0935f883efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:21.786217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-159e2ff3-cda2-42d8-aa54-d0935f883efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:21.897759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e2ff3-cda2-42d8-aa54-d0935f883efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:21.899205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e2ff3-cda2-42d8-aa54-d0935f883efd service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:21.917722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 672/1337] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:24.514126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdd868-674c-43d2-9bd4-53cd61b49bac None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:24.515017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdd868-674c-43d2-9bd4-53cd61b49bac None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:24.515017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdd868-674c-43d2-9bd4-53cd61b49bac None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:24.517208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 666/1338] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:24.525966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cfc60-9a6f-4cfb-9aa3-9460302f6015 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:24.526390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cfc60-9a6f-4cfb-9aa3-9460302f6015 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:24.526617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cfc60-9a6f-4cfb-9aa3-9460302f6015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:24.596104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ba9cfc60-9a6f-4cfb-9aa3-9460302f6015 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:24.597369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ba9cfc60-9a6f-4cfb-9aa3-9460302f6015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:24.703302 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 673/1339] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:24.742503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdd868-674c-43d2-9bd4-53cd61b49bac None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:24.744055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a6fdd868-674c-43d2-9bd4-53cd61b49bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:24.865767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdd868-674c-43d2-9bd4-53cd61b49bac None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:24.943487 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b15d3-4a89-4c61-bcd0-07ef2e01d3fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:24.944133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b15d3-4a89-4c61-bcd0-07ef2e01d3fe service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:24.944409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b15d3-4a89-4c61-bcd0-07ef2e01d3fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:24.944703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b15d3-4a89-4c61-bcd0-07ef2e01d3fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:24.946344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7c9b15d3-4a89-4c61-bcd0-07ef2e01d3fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:25.053008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b15d3-4a89-4c61-bcd0-07ef2e01d3fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:25.054444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b15d3-4a89-4c61-bcd0-07ef2e01d3fe service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:25.073133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 667/1340] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:35:29.452609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e65825-9b38-4042-a3a7-7bd79271ad10 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:29.453498 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e65825-9b38-4042-a3a7-7bd79271ad10 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:29.453742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e65825-9b38-4042-a3a7-7bd79271ad10 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:29.455416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 674/1341] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:29.461508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016df05-7a29-4133-8589-2ca90e5d3f7a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:29.461968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016df05-7a29-4133-8589-2ca90e5d3f7a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:29.462077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016df05-7a29-4133-8589-2ca90e5d3f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:29.535491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f016df05-7a29-4133-8589-2ca90e5d3f7a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:29.536867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f016df05-7a29-4133-8589-2ca90e5d3f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:29.646219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 668/1342] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:29.687260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e65825-9b38-4042-a3a7-7bd79271ad10 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:29.689488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-72e65825-9b38-4042-a3a7-7bd79271ad10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:29.794543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e65825-9b38-4042-a3a7-7bd79271ad10 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:29.798143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c7f1a-05f3-404b-86cc-8a70dbd84bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:29.798761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7f1a-05f3-404b-86cc-8a70dbd84bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:29.799039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7f1a-05f3-404b-86cc-8a70dbd84bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:29.799308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7f1a-05f3-404b-86cc-8a70dbd84bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:29.808706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c7f1a-05f3-404b-86cc-8a70dbd84bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:29.810128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c7f1a-05f3-404b-86cc-8a70dbd84bd5 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:29.819351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 675/1343] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:32.730661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474ebf5-b075-40cd-b4de-b693339ca5e9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:32.731248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474ebf5-b075-40cd-b4de-b693339ca5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:32.731248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474ebf5-b075-40cd-b4de-b693339ca5e9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:32.732568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 669/1344] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:32.738083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d95bf9-976f-4ac5-8cc2-b32b36678aba None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:32.738531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d95bf9-976f-4ac5-8cc2-b32b36678aba None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:32.738650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d95bf9-976f-4ac5-8cc2-b32b36678aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:32.768114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-43d95bf9-976f-4ac5-8cc2-b32b36678aba None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:32.769396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-43d95bf9-976f-4ac5-8cc2-b32b36678aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:32.846795 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 676/1345] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:32.883985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474ebf5-b075-40cd-b4de-b693339ca5e9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:32.896346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474ebf5-b075-40cd-b4de-b693339ca5e9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:32.900523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0e66c-5f8f-4d60-8dbe-16e281fa6e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:32.901175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0e66c-5f8f-4d60-8dbe-16e281fa6e3b service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:32.901484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0e66c-5f8f-4d60-8dbe-16e281fa6e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:32.901753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0e66c-5f8f-4d60-8dbe-16e281fa6e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:32.911293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0e66c-5f8f-4d60-8dbe-16e281fa6e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:32.912796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0e66c-5f8f-4d60-8dbe-16e281fa6e3b service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:32.922286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 670/1346] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:37.404238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b43cf-8127-4b73-96ce-9c9f6f98198c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:37.404872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b43cf-8127-4b73-96ce-9c9f6f98198c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:37.404872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b43cf-8127-4b73-96ce-9c9f6f98198c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:37.406188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 677/1347] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:37.411702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb7b4a-9834-4512-9697-b1fec16e063c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:37.412157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb7b4a-9834-4512-9697-b1fec16e063c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:37.412264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb7b4a-9834-4512-9697-b1fec16e063c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:37.430574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-54fb7b4a-9834-4512-9697-b1fec16e063c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:37.431849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-54fb7b4a-9834-4512-9697-b1fec16e063c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:37.440553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 671/1348] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:37.480254 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b43cf-8127-4b73-96ce-9c9f6f98198c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:37.493076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b43cf-8127-4b73-96ce-9c9f6f98198c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:37.497164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3e424-f706-45db-8afc-83b93b87cdc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:37.497809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3e424-f706-45db-8afc-83b93b87cdc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:37.498090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3e424-f706-45db-8afc-83b93b87cdc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:37.498331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3e424-f706-45db-8afc-83b93b87cdc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:37.508553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3e424-f706-45db-8afc-83b93b87cdc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:37.509994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3e424-f706-45db-8afc-83b93b87cdc9 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:37.519875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 678/1349] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:42.887241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe119d-48c6-48f3-bfaf-9989ef659eb2 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:42.888224 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe119d-48c6-48f3-bfaf-9989ef659eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:42.888724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe119d-48c6-48f3-bfaf-9989ef659eb2 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:42.890565 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 672/1350] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:35:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:42.895245 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e15e7-2df4-4bee-928f-6b167069f8ca None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:42.895653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e15e7-2df4-4bee-928f-6b167069f8ca None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:42.895788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e15e7-2df4-4bee-928f-6b167069f8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:42.969506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-510e15e7-2df4-4bee-928f-6b167069f8ca None None] MFA Rules not processed for user `6e934c8e7dd0425cac68eb91e5e2c0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:42.971294 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-510e15e7-2df4-4bee-928f-6b167069f8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:43.054576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 679/1351] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:35:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:47.184359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86991aaa-87af-4469-b261-769abac2aef4 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:47.185657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86991aaa-87af-4469-b261-769abac2aef4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:47.186288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86991aaa-87af-4469-b261-769abac2aef4 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:47.187959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 673/1352] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:35:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:47.193590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2ac8-6834-4fc3-b2e1-7cc5dbedd377 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:47.194025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2ac8-6834-4fc3-b2e1-7cc5dbedd377 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:47.194148 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2ac8-6834-4fc3-b2e1-7cc5dbedd377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:47.213108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-927a2ac8-6834-4fc3-b2e1-7cc5dbedd377 None None] MFA Rules not processed for user `6e934c8e7dd0425cac68eb91e5e2c0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:47.214407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-927a2ac8-6834-4fc3-b2e1-7cc5dbedd377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:47.222934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 680/1353] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:35:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:48.841509 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4292b-167a-4308-a5db-21156e131f18 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:48.842055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4292b-167a-4308-a5db-21156e131f18 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:48.842225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4292b-167a-4308-a5db-21156e131f18 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:48.843728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 674/1354] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:48.849525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451ffd2-081a-4ea6-849c-e8e948956d3d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:48.849953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451ffd2-081a-4ea6-849c-e8e948956d3d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:48.850104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451ffd2-081a-4ea6-849c-e8e948956d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:48.876887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7451ffd2-081a-4ea6-849c-e8e948956d3d None None] MFA Rules not processed for user `6e934c8e7dd0425cac68eb91e5e2c0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:48.877983 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7451ffd2-081a-4ea6-849c-e8e948956d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:48.887242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 681/1355] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:35:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:52.727139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e71c30-9e33-4abb-a2c6-291f708a48a7 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:52.727726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e71c30-9e33-4abb-a2c6-291f708a48a7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:52.727726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e71c30-9e33-4abb-a2c6-291f708a48a7 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:52.729168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 675/1356] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:52.734591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996f87-a56c-4460-b7a0-e3b3d716e007 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:52.735009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996f87-a56c-4460-b7a0-e3b3d716e007 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:52.735192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996f87-a56c-4460-b7a0-e3b3d716e007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:52.754025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-16996f87-a56c-4460-b7a0-e3b3d716e007 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:52.755239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-16996f87-a56c-4460-b7a0-e3b3d716e007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:52.764120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 682/1357] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:52.814506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cafb6-9574-4ed4-8a28-d3e1292acfc6 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:52.814954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cafb6-9574-4ed4-8a28-d3e1292acfc6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:52.815305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cafb6-9574-4ed4-8a28-d3e1292acfc6 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:52.817262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 676/1358] 10.222.0.50 () {54 vars in 960 bytes} [Fri Apr 29 18:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:52.823662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b042c-af12-49b1-817a-8367cb2afa2e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:52.824309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b042c-af12-49b1-817a-8367cb2afa2e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:52.824808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b042c-af12-49b1-817a-8367cb2afa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:52.874181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9b3b042c-af12-49b1-817a-8367cb2afa2e None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:52.875367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9b3b042c-af12-49b1-817a-8367cb2afa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:52.954169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 683/1359] 10.222.0.50 () {58 vars in 1057 bytes} [Fri Apr 29 18:35:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:52.962750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cafb6-9574-4ed4-8a28-d3e1292acfc6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:52.977503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cafb6-9574-4ed4-8a28-d3e1292acfc6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:52.983423 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13473c7-5fbf-46f6-a229-117585cecfb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:52.984071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13473c7-5fbf-46f6-a229-117585cecfb8 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:52.984353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13473c7-5fbf-46f6-a229-117585cecfb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:52.984835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13473c7-5fbf-46f6-a229-117585cecfb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:52.993396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13473c7-5fbf-46f6-a229-117585cecfb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:52.994997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13473c7-5fbf-46f6-a229-117585cecfb8 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:53.004549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 677/1360] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:55.465579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8ae78-12e6-46e7-8766-0fe2371f6f6f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:55.466141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8ae78-12e6-46e7-8766-0fe2371f6f6f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:55.466141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8ae78-12e6-46e7-8766-0fe2371f6f6f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:55.467866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 684/1361] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:55.472834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abf585-d2ed-48cf-8b48-9661d76cae14 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:55.473427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abf585-d2ed-48cf-8b48-9661d76cae14 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:55.473612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abf585-d2ed-48cf-8b48-9661d76cae14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:55.491847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c1abf585-d2ed-48cf-8b48-9661d76cae14 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:55.493337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c1abf585-d2ed-48cf-8b48-9661d76cae14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:55.501710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 678/1362] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:55.545546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8ae78-12e6-46e7-8766-0fe2371f6f6f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:55.560526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8ae78-12e6-46e7-8766-0fe2371f6f6f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:55.566003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1564f2b-6710-484f-bd86-a24711d57980 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:55.566983 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1564f2b-6710-484f-bd86-a24711d57980 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:55.567178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1564f2b-6710-484f-bd86-a24711d57980 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:55.567559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1564f2b-6710-484f-bd86-a24711d57980 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:55.576229 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1564f2b-6710-484f-bd86-a24711d57980 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:55.578039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1564f2b-6710-484f-bd86-a24711d57980 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:55.588386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 685/1363] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:35:58.165514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18cc28-2af6-4bfc-97e1-6a7e94b11dfa None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:58.166509 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18cc28-2af6-4bfc-97e1-6a7e94b11dfa None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:58.166967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18cc28-2af6-4bfc-97e1-6a7e94b11dfa None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:58.168939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 679/1364] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:35:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:58.174147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdb7f5-7eaf-4f8e-87f5-4253e93d3e35 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:58.174982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdb7f5-7eaf-4f8e-87f5-4253e93d3e35 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:58.175408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdb7f5-7eaf-4f8e-87f5-4253e93d3e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:58.212523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8efdb7f5-7eaf-4f8e-87f5-4253e93d3e35 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:58.214546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8efdb7f5-7eaf-4f8e-87f5-4253e93d3e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:58.232817 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 686/1365] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:35:58.286729 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcac6a-31a2-4a8a-8ee2-126f97ae55aa None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:58.287228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcac6a-31a2-4a8a-8ee2-126f97ae55aa None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:58.287568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcac6a-31a2-4a8a-8ee2-126f97ae55aa None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:58.289017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 680/1366] 10.222.0.50 () {54 vars in 960 bytes} [Fri Apr 29 18:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:35:58.294928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b87fc-803f-4e16-a287-3269ea55640e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:58.295230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b87fc-803f-4e16-a287-3269ea55640e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:58.295500 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b87fc-803f-4e16-a287-3269ea55640e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:58.315210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-be6b87fc-803f-4e16-a287-3269ea55640e None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:35:58.316434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-be6b87fc-803f-4e16-a287-3269ea55640e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:35:58.326904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 687/1367] 10.222.0.50 () {58 vars in 1057 bytes} [Fri Apr 29 18:35:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:35:58.336609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcac6a-31a2-4a8a-8ee2-126f97ae55aa None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:35:58.351900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcac6a-31a2-4a8a-8ee2-126f97ae55aa None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:35:58.357523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026ae92-c4ce-47af-a6cc-06ff2383dab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:35:58.358178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ae92-c4ce-47af-a6cc-06ff2383dab3 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:35:58.358460 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ae92-c4ce-47af-a6cc-06ff2383dab3 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:35:58.358738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ae92-c4ce-47af-a6cc-06ff2383dab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:35:58.367847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ae92-c4ce-47af-a6cc-06ff2383dab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:35:58.369487 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ae92-c4ce-47af-a6cc-06ff2383dab3 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:35:58.379441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 681/1368] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:36:00.719330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689a9a8-55c8-4c2b-a0b5-9d73e3f4382a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:00.719927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689a9a8-55c8-4c2b-a0b5-9d73e3f4382a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:00.719927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689a9a8-55c8-4c2b-a0b5-9d73e3f4382a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:00.721238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 688/1369] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:00.727017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696deb6a-85df-4b3d-a080-8878afac3253 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:00.727518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696deb6a-85df-4b3d-a080-8878afac3253 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:00.727926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696deb6a-85df-4b3d-a080-8878afac3253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:00.747080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-696deb6a-85df-4b3d-a080-8878afac3253 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:00.748746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-696deb6a-85df-4b3d-a080-8878afac3253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:00.757647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 682/1370] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:00.802076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689a9a8-55c8-4c2b-a0b5-9d73e3f4382a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:00.818260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689a9a8-55c8-4c2b-a0b5-9d73e3f4382a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:00.823987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68577be8-0b3f-4e1e-a487-a9e8a10431a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:00.824901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68577be8-0b3f-4e1e-a487-a9e8a10431a1 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:00.825308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68577be8-0b3f-4e1e-a487-a9e8a10431a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:00.825685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68577be8-0b3f-4e1e-a487-a9e8a10431a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:00.836070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68577be8-0b3f-4e1e-a487-a9e8a10431a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:00.837912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68577be8-0b3f-4e1e-a487-a9e8a10431a1 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:00.849185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 689/1371] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:00.946717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c3ecb-686d-43ae-ad4d-9d44c86befb1 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:00.947178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c3ecb-686d-43ae-ad4d-9d44c86befb1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:00.947581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c3ecb-686d-43ae-ad4d-9d44c86befb1 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:00.949949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 683/1372] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:00.955817 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14849d16-160c-4568-9c48-d5357fb41c29 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:00.956102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14849d16-160c-4568-9c48-d5357fb41c29 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:00.956321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14849d16-160c-4568-9c48-d5357fb41c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:00.977135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-14849d16-160c-4568-9c48-d5357fb41c29 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:00.978839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-14849d16-160c-4568-9c48-d5357fb41c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:00.989362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 690/1373] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:01.295706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e1630-202c-4b1d-a0dc-b50e26ece32c None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:01.296156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e1630-202c-4b1d-a0dc-b50e26ece32c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:01.296326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e1630-202c-4b1d-a0dc-b50e26ece32c None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:01.297724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 684/1374] 10.222.0.50 () {54 vars in 962 bytes} [Fri Apr 29 18:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:01.303189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c78ba-4ba2-44f6-b0ff-a8dc343123e4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:01.303567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c78ba-4ba2-44f6-b0ff-a8dc343123e4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:01.303912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c78ba-4ba2-44f6-b0ff-a8dc343123e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:01.357251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d58c78ba-4ba2-44f6-b0ff-a8dc343123e4 None None] MFA Rules not processed for user `ae4fe640bb564a8abe872cb1ec74dc7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:01.358721 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d58c78ba-4ba2-44f6-b0ff-a8dc343123e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:01.419486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 691/1375] 10.222.0.50 () {58 vars in 1059 bytes} [Fri Apr 29 18:36:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:01.427523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e1630-202c-4b1d-a0dc-b50e26ece32c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:01.438251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e1630-202c-4b1d-a0dc-b50e26ece32c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:01.441914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa814ad-51d5-4422-8fa7-5373fdf91042 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:01.442717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa814ad-51d5-4422-8fa7-5373fdf91042 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:01.442994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa814ad-51d5-4422-8fa7-5373fdf91042 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:01.443266 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa814ad-51d5-4422-8fa7-5373fdf91042 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:01.452248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa814ad-51d5-4422-8fa7-5373fdf91042 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:01.453722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa814ad-51d5-4422-8fa7-5373fdf91042 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:01.463945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 685/1376] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:36:03.119258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968b3a6-4665-4e2a-b85d-b35a5a76e9f0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:03.119870 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968b3a6-4665-4e2a-b85d-b35a5a76e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:03.119870 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968b3a6-4665-4e2a-b85d-b35a5a76e9f0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:03.121228 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 692/1377] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:03.127165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76476d9f-1c3f-49d2-b12c-c0b3a04d3c8e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:03.127774 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76476d9f-1c3f-49d2-b12c-c0b3a04d3c8e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:03.127774 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76476d9f-1c3f-49d2-b12c-c0b3a04d3c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:03.147344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-76476d9f-1c3f-49d2-b12c-c0b3a04d3c8e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:03.148600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-76476d9f-1c3f-49d2-b12c-c0b3a04d3c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:03.158588 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 686/1378] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:03.202237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968b3a6-4665-4e2a-b85d-b35a5a76e9f0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:03.217645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968b3a6-4665-4e2a-b85d-b35a5a76e9f0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:03.223697 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6974f36-57af-4eb0-abd7-3d5d263f1fc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:03.224376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6974f36-57af-4eb0-abd7-3d5d263f1fc1 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:03.224733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6974f36-57af-4eb0-abd7-3d5d263f1fc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:03.225288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6974f36-57af-4eb0-abd7-3d5d263f1fc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:03.233898 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6974f36-57af-4eb0-abd7-3d5d263f1fc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:03.235506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6974f36-57af-4eb0-abd7-3d5d263f1fc1 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:03.245135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 693/1379] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:05.517528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad337b2b-ce37-4281-b6d7-909a680d8ba7 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:05.518124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad337b2b-ce37-4281-b6d7-909a680d8ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:05.518124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad337b2b-ce37-4281-b6d7-909a680d8ba7 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:05.519405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 687/1380] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:05.525046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376678ea-f21a-4876-a914-4580449dd7f8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:05.525485 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376678ea-f21a-4876-a914-4580449dd7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:05.525615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376678ea-f21a-4876-a914-4580449dd7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:05.543816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-376678ea-f21a-4876-a914-4580449dd7f8 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:05.545049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-376678ea-f21a-4876-a914-4580449dd7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:05.553922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 694/1381] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:05.601451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad337b2b-ce37-4281-b6d7-909a680d8ba7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:05.617043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad337b2b-ce37-4281-b6d7-909a680d8ba7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:05.627093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d8e9e-ed03-41c2-97ac-ead0cf7d2681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:05.627862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8e9e-ed03-41c2-97ac-ead0cf7d2681 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:05.628235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8e9e-ed03-41c2-97ac-ead0cf7d2681 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:05.628635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8e9e-ed03-41c2-97ac-ead0cf7d2681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:05.638102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d8e9e-ed03-41c2-97ac-ead0cf7d2681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:05.639787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d8e9e-ed03-41c2-97ac-ead0cf7d2681 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:05.649846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 688/1382] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:07.916535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3249c0-0641-4e9a-9ae9-16814290830f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:07.917100 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3249c0-0641-4e9a-9ae9-16814290830f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:07.917307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3249c0-0641-4e9a-9ae9-16814290830f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:07.918811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 695/1383] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:07.924209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686243b-4608-4fbb-9710-87aebbfd9059 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:07.924789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686243b-4608-4fbb-9710-87aebbfd9059 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:07.924868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686243b-4608-4fbb-9710-87aebbfd9059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:07.944931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e686243b-4608-4fbb-9710-87aebbfd9059 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:07.946252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e686243b-4608-4fbb-9710-87aebbfd9059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:07.954700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 689/1384] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:07.998041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3249c0-0641-4e9a-9ae9-16814290830f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:08.015717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3249c0-0641-4e9a-9ae9-16814290830f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:08.021526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01dcb1-bada-4d02-87a7-e663d8082d15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:08.022299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01dcb1-bada-4d02-87a7-e663d8082d15 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:08.022768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01dcb1-bada-4d02-87a7-e663d8082d15 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:08.023133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01dcb1-bada-4d02-87a7-e663d8082d15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:08.031570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01dcb1-bada-4d02-87a7-e663d8082d15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:08.033382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01dcb1-bada-4d02-87a7-e663d8082d15 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:08.043548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 696/1385] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:10.639405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cea5fd-3768-4de8-9054-ef2eb2cd3e05 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:10.640040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cea5fd-3768-4de8-9054-ef2eb2cd3e05 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:10.640040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cea5fd-3768-4de8-9054-ef2eb2cd3e05 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:10.641653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 690/1386] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:10.649170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05bf91-d7de-44de-afbe-effba74216cc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:10.649691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05bf91-d7de-44de-afbe-effba74216cc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:10.649691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05bf91-d7de-44de-afbe-effba74216cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:10.672203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2f05bf91-d7de-44de-afbe-effba74216cc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:10.673695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2f05bf91-d7de-44de-afbe-effba74216cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:10.683199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 697/1387] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:10.731453 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cea5fd-3768-4de8-9054-ef2eb2cd3e05 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:10.746653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cea5fd-3768-4de8-9054-ef2eb2cd3e05 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:10.752213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc0e68-b68a-4efc-b9c7-a650889e1243 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:10.752920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc0e68-b68a-4efc-b9c7-a650889e1243 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:10.753200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc0e68-b68a-4efc-b9c7-a650889e1243 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:10.753482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc0e68-b68a-4efc-b9c7-a650889e1243 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:10.762292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc0e68-b68a-4efc-b9c7-a650889e1243 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:10.763959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc0e68-b68a-4efc-b9c7-a650889e1243 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:10.773680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 691/1388] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:13.353526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced920f-1520-4f2f-9004-ccabe394d1a9 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:13.354172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced920f-1520-4f2f-9004-ccabe394d1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:13.354172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced920f-1520-4f2f-9004-ccabe394d1a9 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:13.356134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 698/1389] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:13.362143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca276c-ffba-48fe-a94a-b766285770d5 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:13.362636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca276c-ffba-48fe-a94a-b766285770d5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:13.362738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca276c-ffba-48fe-a94a-b766285770d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:13.384702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-71ca276c-ffba-48fe-a94a-b766285770d5 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:13.385948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-71ca276c-ffba-48fe-a94a-b766285770d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:13.395757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 692/1390] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:13.441063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced920f-1520-4f2f-9004-ccabe394d1a9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:13.457711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced920f-1520-4f2f-9004-ccabe394d1a9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:13.464114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd14349-714b-4e2a-b51e-ac7715bddf5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:13.464847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd14349-714b-4e2a-b51e-ac7715bddf5d service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:13.465126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd14349-714b-4e2a-b51e-ac7715bddf5d service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:13.465393 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd14349-714b-4e2a-b51e-ac7715bddf5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:13.474811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd14349-714b-4e2a-b51e-ac7715bddf5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:13.476449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd14349-714b-4e2a-b51e-ac7715bddf5d service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:13.486049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 699/1391] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:15.792697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5149a1-0913-4b0a-abad-848f39369ce4 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:15.793342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5149a1-0913-4b0a-abad-848f39369ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:15.793342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5149a1-0913-4b0a-abad-848f39369ce4 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:15.794596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 693/1392] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:15.800077 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645746-1ccf-44b0-808b-f8d1d3d29fa3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:15.800566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645746-1ccf-44b0-808b-f8d1d3d29fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:15.800669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645746-1ccf-44b0-808b-f8d1d3d29fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:15.819502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-63645746-1ccf-44b0-808b-f8d1d3d29fa3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:15.820745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-63645746-1ccf-44b0-808b-f8d1d3d29fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:15.829588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 700/1393] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:15.873840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5149a1-0913-4b0a-abad-848f39369ce4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:15.889229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5149a1-0913-4b0a-abad-848f39369ce4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:15.896463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2bd51-990f-4a25-9ad5-bb2b25b26d90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:15.897213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2bd51-990f-4a25-9ad5-bb2b25b26d90 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:15.897559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2bd51-990f-4a25-9ad5-bb2b25b26d90 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:15.897902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2bd51-990f-4a25-9ad5-bb2b25b26d90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:15.906843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2bd51-990f-4a25-9ad5-bb2b25b26d90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:15.908431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2bd51-990f-4a25-9ad5-bb2b25b26d90 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:15.918138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 694/1394] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:17.198326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b5c76-12bc-409c-afa7-2291f748a2d0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:17.198927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b5c76-12bc-409c-afa7-2291f748a2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:17.198927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b5c76-12bc-409c-afa7-2291f748a2d0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:17.200236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 701/1395] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:17.205587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5acbca-9684-4b8d-a375-fd4d48c08c92 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:17.206062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5acbca-9684-4b8d-a375-fd4d48c08c92 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:17.206165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5acbca-9684-4b8d-a375-fd4d48c08c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:17.225753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bd5acbca-9684-4b8d-a375-fd4d48c08c92 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:17.226953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bd5acbca-9684-4b8d-a375-fd4d48c08c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:17.236295 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 695/1396] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:17.291292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632fac6-1aaf-4fe3-8324-d90b44fd98ee None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:17.291577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632fac6-1aaf-4fe3-8324-d90b44fd98ee None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:17.291853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632fac6-1aaf-4fe3-8324-d90b44fd98ee None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:17.293264 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 702/1397] 10.222.0.50 () {54 vars in 962 bytes} [Fri Apr 29 18:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:17.298224 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587b85-cd24-41fe-8263-b8371c4638d4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:17.298606 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587b85-cd24-41fe-8263-b8371c4638d4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:17.298942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587b85-cd24-41fe-8263-b8371c4638d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:17.349757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a8587b85-cd24-41fe-8263-b8371c4638d4 None None] MFA Rules not processed for user `6552c41f1c654ef5bfdd940e3b807dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:17.350951 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a8587b85-cd24-41fe-8263-b8371c4638d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:17.411080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 696/1398] 10.222.0.50 () {58 vars in 1059 bytes} [Fri Apr 29 18:36:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:17.418920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632fac6-1aaf-4fe3-8324-d90b44fd98ee None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:17.430650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632fac6-1aaf-4fe3-8324-d90b44fd98ee None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:17.434719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd47b8-c231-41e1-8be4-ea6c1cf2b5d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6552c41f1c654ef5bfdd940e3b807dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:17.435365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd47b8-c231-41e1-8be4-ea6c1cf2b5d1 service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:17.435645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd47b8-c231-41e1-8be4-ea6c1cf2b5d1 service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:17.435917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd47b8-c231-41e1-8be4-ea6c1cf2b5d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:17.445042 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd47b8-c231-41e1-8be4-ea6c1cf2b5d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:17.446476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd47b8-c231-41e1-8be4-ea6c1cf2b5d1 service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:17.456157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 703/1399] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:17.877225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafacea9-90d6-4183-9f00-a9e74039ac3c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:17.877510 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafacea9-90d6-4183-9f00-a9e74039ac3c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:17.877777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafacea9-90d6-4183-9f00-a9e74039ac3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:17.927928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dafacea9-90d6-4183-9f00-a9e74039ac3c None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:17.929152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dafacea9-90d6-4183-9f00-a9e74039ac3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:17.994396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 697/1400] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:36:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:18.038805 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdaaf9-4628-4ded-95fb-1aec2ebcfb52 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:18.039080 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdaaf9-4628-4ded-95fb-1aec2ebcfb52 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:18.039375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdaaf9-4628-4ded-95fb-1aec2ebcfb52 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:18.040858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 704/1401] 10.222.0.50 () {54 vars in 942 bytes} [Fri Apr 29 18:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:18.047994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e2526-13a1-45de-b4b0-04761279b309 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:18.048284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e2526-13a1-45de-b4b0-04761279b309 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:18.048594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e2526-13a1-45de-b4b0-04761279b309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:18.099878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6d4e2526-13a1-45de-b4b0-04761279b309 None None] MFA Rules not processed for user `4cf46333cd034f5dbd0b78503de1f8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:18.101117 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6d4e2526-13a1-45de-b4b0-04761279b309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:18.175378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 698/1402] 10.222.0.50 () {58 vars in 1039 bytes} [Fri Apr 29 18:36:18 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:18.185483 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdaaf9-4628-4ded-95fb-1aec2ebcfb52 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:18.199507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdaaf9-4628-4ded-95fb-1aec2ebcfb52 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:18.205259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdaa18-5eb7-4654-9c57-50a23abcddc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:18.205902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdaa18-5eb7-4654-9c57-50a23abcddc4 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:18.206178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdaa18-5eb7-4654-9c57-50a23abcddc4 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:18.206421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdaa18-5eb7-4654-9c57-50a23abcddc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:18.215369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bdaa18-5eb7-4654-9c57-50a23abcddc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:18.217080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bba32e-dfe8-40d8-a55c-00af85ceb722 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:18.217524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bba32e-dfe8-40d8-a55c-00af85ceb722 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:18.217620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bba32e-dfe8-40d8-a55c-00af85ceb722 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:18.218053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bdaa18-5eb7-4654-9c57-50a23abcddc4 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:18.219120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 699/1403] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:36:18.224137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc602e-8ce0-45fd-8294-1ea41c31a2f3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:18.224433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc602e-8ce0-45fd-8294-1ea41c31a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:18.224761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc602e-8ce0-45fd-8294-1ea41c31a2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:18.227088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 705/1404] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:36:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:36:18.244285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f9cc602e-8ce0-45fd-8294-1ea41c31a2f3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:18.245570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f9cc602e-8ce0-45fd-8294-1ea41c31a2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:18.253833 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 700/1405] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:18.299698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdaa18-5eb7-4654-9c57-50a23abcddc4 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:18.311944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdaa18-5eb7-4654-9c57-50a23abcddc4 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:18.318930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec656221-3ba6-442f-81c8-984f61742d7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:18.318930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec656221-3ba6-442f-81c8-984f61742d7a service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:18.318930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec656221-3ba6-442f-81c8-984f61742d7a service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:18.319536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec656221-3ba6-442f-81c8-984f61742d7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:18.336361 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec656221-3ba6-442f-81c8-984f61742d7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:18.338246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec656221-3ba6-442f-81c8-984f61742d7a service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:18.349797 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 706/1406] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:20.665097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16c1b3-9a00-49d2-aa79-eb4e269d1edf None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:20.665638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16c1b3-9a00-49d2-aa79-eb4e269d1edf None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:20.665638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16c1b3-9a00-49d2-aa79-eb4e269d1edf None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:20.667010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 701/1407] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:20.672154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b859d-3773-42a4-b795-7af65323d55d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:20.672660 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b859d-3773-42a4-b795-7af65323d55d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:20.672754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b859d-3773-42a4-b795-7af65323d55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:20.691417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-499b859d-3773-42a4-b795-7af65323d55d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:20.692885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-499b859d-3773-42a4-b795-7af65323d55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:20.701212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 707/1408] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:20.746099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16c1b3-9a00-49d2-aa79-eb4e269d1edf None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:20.764612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16c1b3-9a00-49d2-aa79-eb4e269d1edf None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:20.767132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f2a5c-5ba0-4f58-83f4-211c96e91b81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:20.767922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f2a5c-5ba0-4f58-83f4-211c96e91b81 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:20.768374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f2a5c-5ba0-4f58-83f4-211c96e91b81 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:20.768683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f2a5c-5ba0-4f58-83f4-211c96e91b81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:20.777456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f2a5c-5ba0-4f58-83f4-211c96e91b81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:20.779052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f2a5c-5ba0-4f58-83f4-211c96e91b81 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:20.788449 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 702/1409] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:41.616194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b8622-db84-4fd8-aa64-1b21937746e6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:41.616881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b8622-db84-4fd8-aa64-1b21937746e6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:41.616881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b8622-db84-4fd8-aa64-1b21937746e6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:41.618176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 708/1410] 10.222.0.50 () {54 vars in 918 bytes} [Fri Apr 29 18:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:41.623849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd09a0-f58e-44c1-89c6-a169285ba9e8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:41.624352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd09a0-f58e-44c1-89c6-a169285ba9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:41.624352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd09a0-f58e-44c1-89c6-a169285ba9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:41.643618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f3dd09a0-f58e-44c1-89c6-a169285ba9e8 None None] MFA Rules not processed for user `6e934c8e7dd0425cac68eb91e5e2c0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:41.644962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f3dd09a0-f58e-44c1-89c6-a169285ba9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:41.654907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 703/1411] 10.222.0.50 () {58 vars in 1015 bytes} [Fri Apr 29 18:36:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:41.669137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c60e1-91e4-4b02-9231-8a759e615ee8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:41.669444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c60e1-91e4-4b02-9231-8a759e615ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:41.669713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c60e1-91e4-4b02-9231-8a759e615ee8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:41.671086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 709/1412] 10.222.0.50 () {54 vars in 918 bytes} [Fri Apr 29 18:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:41.676147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67523a-4fc3-4e88-81a0-2d0c99824108 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:41.676426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67523a-4fc3-4e88-81a0-2d0c99824108 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:41.676720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67523a-4fc3-4e88-81a0-2d0c99824108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:41.695310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7a67523a-4fc3-4e88-81a0-2d0c99824108 None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:41.696499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7a67523a-4fc3-4e88-81a0-2d0c99824108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:41.706284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 704/1413] 10.222.0.50 () {58 vars in 1015 bytes} [Fri Apr 29 18:36:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:44.428167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782a2c5-e93b-4756-ad00-aae0e25e1955 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:44.428764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782a2c5-e93b-4756-ad00-aae0e25e1955 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:44.428764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782a2c5-e93b-4756-ad00-aae0e25e1955 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:44.430838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:36:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:44.439611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb930a-908d-494e-b13a-7504c875755a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:44.440043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb930a-908d-494e-b13a-7504c875755a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:44.440224 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb930a-908d-494e-b13a-7504c875755a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:44.465603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f7bb930a-908d-494e-b13a-7504c875755a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:44.466814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f7bb930a-908d-494e-b13a-7504c875755a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:44.476154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:44.521692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782a2c5-e93b-4756-ad00-aae0e25e1955 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:44.546986 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782a2c5-e93b-4756-ad00-aae0e25e1955 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:44.558233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ef7ab-3620-4ed1-905c-bab20321a4e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:44.559133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ef7ab-3620-4ed1-905c-bab20321a4e9 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:44.559476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ef7ab-3620-4ed1-905c-bab20321a4e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:44.559801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ef7ab-3620-4ed1-905c-bab20321a4e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:44.572837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ef7ab-3620-4ed1-905c-bab20321a4e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:44.575116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ef7ab-3620-4ed1-905c-bab20321a4e9 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:44.587651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 711/1416] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:45.878238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0718-b4f8-43cd-a3b6-87b50abd60f6 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:45.879217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0718-b4f8-43cd-a3b6-87b50abd60f6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:45.879217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0718-b4f8-43cd-a3b6-87b50abd60f6 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:45.881460 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:36:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:45.889702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874be49-6458-4168-a269-bf7fdc269703 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:45.890168 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874be49-6458-4168-a269-bf7fdc269703 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:45.890263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874be49-6458-4168-a269-bf7fdc269703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:45.911413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6874be49-6458-4168-a269-bf7fdc269703 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:45.912512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6874be49-6458-4168-a269-bf7fdc269703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:45.922098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:46.078845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca0718-b4f8-43cd-a3b6-87b50abd60f6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:46.097850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca0718-b4f8-43cd-a3b6-87b50abd60f6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:46.104530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e28890-8a56-4df8-a621-9567d61a69ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:46.105181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28890-8a56-4df8-a621-9567d61a69ce service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:46.105460 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28890-8a56-4df8-a621-9567d61a69ce service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:46.105733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28890-8a56-4df8-a621-9567d61a69ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:46.114446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e28890-8a56-4df8-a621-9567d61a69ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:46.116024 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e28890-8a56-4df8-a621-9567d61a69ce service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:46.125334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 707/1419] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:47.404001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb20fc-61a9-4e9d-ba6b-9c89cf4edf57 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:47.404830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb20fc-61a9-4e9d-ba6b-9c89cf4edf57 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:47.404830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb20fc-61a9-4e9d-ba6b-9c89cf4edf57 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:47.406541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:36:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:47.414404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804db6ca-005d-4bbf-b747-6937853fe3da None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:47.414818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804db6ca-005d-4bbf-b747-6937853fe3da None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:47.414944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804db6ca-005d-4bbf-b747-6937853fe3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:47.438019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-804db6ca-005d-4bbf-b747-6937853fe3da None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:47.439197 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-804db6ca-005d-4bbf-b747-6937853fe3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:47.450528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:47.607990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb20fc-61a9-4e9d-ba6b-9c89cf4edf57 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:47.626538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb20fc-61a9-4e9d-ba6b-9c89cf4edf57 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:47.633134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e28fc2-53eb-40cb-be53-31b7eb0fb8f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:47.633785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e28fc2-53eb-40cb-be53-31b7eb0fb8f2 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:47.634057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e28fc2-53eb-40cb-be53-31b7eb0fb8f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:47.634325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e28fc2-53eb-40cb-be53-31b7eb0fb8f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:47.643491 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e28fc2-53eb-40cb-be53-31b7eb0fb8f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:47.645353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e28fc2-53eb-40cb-be53-31b7eb0fb8f2 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:47.655042 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 714/1422] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:48.898894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a534b-7b1e-4dd1-b10e-32d0731102e2 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:48.899749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a534b-7b1e-4dd1-b10e-32d0731102e2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:48.899749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a534b-7b1e-4dd1-b10e-32d0731102e2 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:48.901899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:36:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:48.909236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb804c5e-f82f-4391-964c-1f5c88882a2c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:48.909687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb804c5e-f82f-4391-964c-1f5c88882a2c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:48.909780 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb804c5e-f82f-4391-964c-1f5c88882a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:48.929667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fb804c5e-f82f-4391-964c-1f5c88882a2c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:48.930863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fb804c5e-f82f-4391-964c-1f5c88882a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:48.939111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:36:49.087408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a534b-7b1e-4dd1-b10e-32d0731102e2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:49.103412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a534b-7b1e-4dd1-b10e-32d0731102e2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:49.111559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a92345-9a11-4cc2-96c8-b958dd57a9f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:49.112559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a92345-9a11-4cc2-96c8-b958dd57a9f2 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:49.113047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a92345-9a11-4cc2-96c8-b958dd57a9f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:49.113872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a92345-9a11-4cc2-96c8-b958dd57a9f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:49.122912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a92345-9a11-4cc2-96c8-b958dd57a9f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:49.124534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a92345-9a11-4cc2-96c8-b958dd57a9f2 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:49.134682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 710/1425] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:50.392907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a680e-7766-4216-a439-b2317792a783 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:50.393490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a680e-7766-4216-a439-b2317792a783 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:50.393490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a680e-7766-4216-a439-b2317792a783 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:50.394982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:50.401461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea80f-99d8-4d08-a6d2-ac785c659428 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:50.401922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea80f-99d8-4d08-a6d2-ac785c659428 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:50.402014 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea80f-99d8-4d08-a6d2-ac785c659428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:50.427652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-057ea80f-99d8-4d08-a6d2-ac785c659428 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:50.428898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-057ea80f-99d8-4d08-a6d2-ac785c659428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:50.439073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:50.596086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a680e-7766-4216-a439-b2317792a783 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:50.617633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a680e-7766-4216-a439-b2317792a783 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:50.624691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b14fd-2b4a-4a83-901b-fdd9ef677cb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:50.625345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b14fd-2b4a-4a83-901b-fdd9ef677cb0 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:50.625625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b14fd-2b4a-4a83-901b-fdd9ef677cb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:50.625892 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b14fd-2b4a-4a83-901b-fdd9ef677cb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:50.635412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b14fd-2b4a-4a83-901b-fdd9ef677cb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:50.637050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b14fd-2b4a-4a83-901b-fdd9ef677cb0 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:50.647102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 717/1428] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:51.854491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f5c58-7086-4551-9c4b-72da1bc1a604 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:51.855140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f5c58-7086-4551-9c4b-72da1bc1a604 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:51.855140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f5c58-7086-4551-9c4b-72da1bc1a604 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:51.856380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:36:51.862681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba42ecd-cb6a-4046-9697-5f7c33ac22ed None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:51.863548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba42ecd-cb6a-4046-9697-5f7c33ac22ed None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:51.863548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba42ecd-cb6a-4046-9697-5f7c33ac22ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:51.884416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cba42ecd-cb6a-4046-9697-5f7c33ac22ed None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:51.885582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cba42ecd-cb6a-4046-9697-5f7c33ac22ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:51.895334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:52.051532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f5c58-7086-4551-9c4b-72da1bc1a604 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:52.077307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f5c58-7086-4551-9c4b-72da1bc1a604 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:52.086353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a216f6-dedc-4846-bdf3-136ff4fb4d5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:52.086995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a216f6-dedc-4846-bdf3-136ff4fb4d5d service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:52.087271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a216f6-dedc-4846-bdf3-136ff4fb4d5d service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:52.087541 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a216f6-dedc-4846-bdf3-136ff4fb4d5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:52.095649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a216f6-dedc-4846-bdf3-136ff4fb4d5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:52.097259 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a216f6-dedc-4846-bdf3-136ff4fb4d5d service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:52.106501 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 713/1431] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:53.295823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededc20b-6a19-453b-b25e-73b1a6d215fd None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:53.296379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededc20b-6a19-453b-b25e-73b1a6d215fd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:53.296379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededc20b-6a19-453b-b25e-73b1a6d215fd None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:53.297681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:53.303690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074912f-83ac-4a52-9bd3-5d634b9310d0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:53.304140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074912f-83ac-4a52-9bd3-5d634b9310d0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:53.304245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074912f-83ac-4a52-9bd3-5d634b9310d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:53.326726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0074912f-83ac-4a52-9bd3-5d634b9310d0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:53.329716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0074912f-83ac-4a52-9bd3-5d634b9310d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:53.341123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:53.496127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededc20b-6a19-453b-b25e-73b1a6d215fd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:53.519929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededc20b-6a19-453b-b25e-73b1a6d215fd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:53.528602 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f934398-cb15-49da-a34a-65974a87f47b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:53.529503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f934398-cb15-49da-a34a-65974a87f47b service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:53.530078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f934398-cb15-49da-a34a-65974a87f47b service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:53.530436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f934398-cb15-49da-a34a-65974a87f47b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:53.543450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f934398-cb15-49da-a34a-65974a87f47b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:53.545942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f934398-cb15-49da-a34a-65974a87f47b service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:53.558217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 720/1434] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:54.740517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63760f4-4510-4965-9d7f-3a2a55797aa4 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:54.741088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63760f4-4510-4965-9d7f-3a2a55797aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:54.741088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63760f4-4510-4965-9d7f-3a2a55797aa4 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:54.742407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:54.748606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabaa33-1df7-4618-bc84-e2044b97f4da None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:54.749078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabaa33-1df7-4618-bc84-e2044b97f4da None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:54.749185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabaa33-1df7-4618-bc84-e2044b97f4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:54.770182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7cabaa33-1df7-4618-bc84-e2044b97f4da None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:54.771372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7cabaa33-1df7-4618-bc84-e2044b97f4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:54.780854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:54.930481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63760f4-4510-4965-9d7f-3a2a55797aa4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:54.956728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63760f4-4510-4965-9d7f-3a2a55797aa4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:54.964703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a0c90-2996-4363-b8a6-fe669056cffa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:54.965343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a0c90-2996-4363-b8a6-fe669056cffa service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:54.965623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a0c90-2996-4363-b8a6-fe669056cffa service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:54.965894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a0c90-2996-4363-b8a6-fe669056cffa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:54.974268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a0c90-2996-4363-b8a6-fe669056cffa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:54.975844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a0c90-2996-4363-b8a6-fe669056cffa service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:54.985879 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 716/1437] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:56.176182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282e9b2-030a-4c93-b799-b1ca244cf7ce None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:56.176800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282e9b2-030a-4c93-b799-b1ca244cf7ce None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:56.176800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282e9b2-030a-4c93-b799-b1ca244cf7ce None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:56.178075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:56.184347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02f98b-a74b-450c-a17d-00b64cfad014 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:56.185154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02f98b-a74b-450c-a17d-00b64cfad014 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:56.185154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02f98b-a74b-450c-a17d-00b64cfad014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:56.205820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6c02f98b-a74b-450c-a17d-00b64cfad014 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:56.207299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6c02f98b-a74b-450c-a17d-00b64cfad014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:56.216256 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:56.365472 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4282e9b2-030a-4c93-b799-b1ca244cf7ce None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:56.380918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4282e9b2-030a-4c93-b799-b1ca244cf7ce None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:56.387002 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576652ac-c1b2-43eb-8353-b71862846eb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:56.387637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576652ac-c1b2-43eb-8353-b71862846eb5 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:56.387915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576652ac-c1b2-43eb-8353-b71862846eb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:56.388190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576652ac-c1b2-43eb-8353-b71862846eb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:56.396446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576652ac-c1b2-43eb-8353-b71862846eb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:56.398047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576652ac-c1b2-43eb-8353-b71862846eb5 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:56.409031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 723/1440] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:36:58.552380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0adda5-881e-4c2c-9898-999e376e97b1 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:58.553023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0adda5-881e-4c2c-9898-999e376e97b1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:58.553023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0adda5-881e-4c2c-9898-999e376e97b1 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:58.554332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:36:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:36:58.559641 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90056ccb-e2ed-47be-bd6b-505748b598ab None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:58.560088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90056ccb-e2ed-47be-bd6b-505748b598ab None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:58.560194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90056ccb-e2ed-47be-bd6b-505748b598ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:58.582090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-90056ccb-e2ed-47be-bd6b-505748b598ab None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:36:58.583387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-90056ccb-e2ed-47be-bd6b-505748b598ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:36:58.592918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:36:58.635719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0adda5-881e-4c2c-9898-999e376e97b1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:36:58.649580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0adda5-881e-4c2c-9898-999e376e97b1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:36:58.656786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985e7cc-03b4-4086-8af5-1bf3e381a996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:36:58.657745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985e7cc-03b4-4086-8af5-1bf3e381a996 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:36:58.658130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985e7cc-03b4-4086-8af5-1bf3e381a996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:36:58.658475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985e7cc-03b4-4086-8af5-1bf3e381a996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:36:58.669190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985e7cc-03b4-4086-8af5-1bf3e381a996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:36:58.670610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985e7cc-03b4-4086-8af5-1bf3e381a996 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:36:58.680039 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 719/1443] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:37:01.192614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afcacf-d182-4bd7-aeec-074707d6c13d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:01.193669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afcacf-d182-4bd7-aeec-074707d6c13d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:01.193669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afcacf-d182-4bd7-aeec-074707d6c13d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:01.195723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:37:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:01.202192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735905a3-2109-4112-a596-31315a78b5b0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:01.202654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735905a3-2109-4112-a596-31315a78b5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:01.202760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735905a3-2109-4112-a596-31315a78b5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:01.229933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-735905a3-2109-4112-a596-31315a78b5b0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:01.231115 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-735905a3-2109-4112-a596-31315a78b5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:01.240201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:37:01.282034 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afcacf-d182-4bd7-aeec-074707d6c13d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:01.295479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afcacf-d182-4bd7-aeec-074707d6c13d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:01.299946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c297d0-0ba1-4423-8773-d76a2424b87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:01.300772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c297d0-0ba1-4423-8773-d76a2424b87b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:01.300772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c297d0-0ba1-4423-8773-d76a2424b87b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:01.301338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c297d0-0ba1-4423-8773-d76a2424b87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:01.310895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c297d0-0ba1-4423-8773-d76a2424b87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:01.312314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c297d0-0ba1-4423-8773-d76a2424b87b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:01.321957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 726/1446] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:37:03.921790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d45d15-d185-49f1-9b27-46fff25b9ebf None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:03.922270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d45d15-d185-49f1-9b27-46fff25b9ebf None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:03.922505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d45d15-d185-49f1-9b27-46fff25b9ebf None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:03.924383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:37:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:03.930480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113a508-5998-4765-a6df-5c319d18fbc1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:03.930970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113a508-5998-4765-a6df-5c319d18fbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:03.930970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113a508-5998-4765-a6df-5c319d18fbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:03.949508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f113a508-5998-4765-a6df-5c319d18fbc1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:03.950692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f113a508-5998-4765-a6df-5c319d18fbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:03.959070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:37:03.999865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d45d15-d185-49f1-9b27-46fff25b9ebf None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:04.002849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-70d45d15-d185-49f1-9b27-46fff25b9ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:04.023326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d45d15-d185-49f1-9b27-46fff25b9ebf None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:04.027556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7d4cf-7e69-4ae6-9211-7e6d1150c78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:04.028231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d4cf-7e69-4ae6-9211-7e6d1150c78d service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:04.028528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d4cf-7e69-4ae6-9211-7e6d1150c78d service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:04.028790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d4cf-7e69-4ae6-9211-7e6d1150c78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:04.039930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7d4cf-7e69-4ae6-9211-7e6d1150c78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:04.041419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7d4cf-7e69-4ae6-9211-7e6d1150c78d service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:04.051993 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 722/1449] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:37:06.404371 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998cbde-9447-4fdf-9753-e02a8bf0c394 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:06.405019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998cbde-9447-4fdf-9753-e02a8bf0c394 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:06.405019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998cbde-9447-4fdf-9753-e02a8bf0c394 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:06.406345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:06.412558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9b0d-d68d-4ed9-b332-7657c8092444 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:06.412988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9b0d-d68d-4ed9-b332-7657c8092444 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:06.413106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9b0d-d68d-4ed9-b332-7657c8092444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:06.442336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b95d9b0d-d68d-4ed9-b332-7657c8092444 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:06.443539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b95d9b0d-d68d-4ed9-b332-7657c8092444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:06.456468 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:37:06.494944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998cbde-9447-4fdf-9753-e02a8bf0c394 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:06.510093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998cbde-9447-4fdf-9753-e02a8bf0c394 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:06.514948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c43153-d79e-48c1-b04d-2c12c0908c96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:06.515586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c43153-d79e-48c1-b04d-2c12c0908c96 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:06.515861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c43153-d79e-48c1-b04d-2c12c0908c96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:06.516124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c43153-d79e-48c1-b04d-2c12c0908c96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:06.527172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c43153-d79e-48c1-b04d-2c12c0908c96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:06.529419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c43153-d79e-48c1-b04d-2c12c0908c96 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:06.541898 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 729/1452] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:37:08.818746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebad46d-845a-49f8-a081-bcba9e479e4d None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:08.819311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebad46d-845a-49f8-a081-bcba9e479e4d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:08.819311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebad46d-845a-49f8-a081-bcba9e479e4d None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:08.821697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:37:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:08.829910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cac039-4c3f-4810-99da-075598b7ff0b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:08.830382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cac039-4c3f-4810-99da-075598b7ff0b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:08.830490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cac039-4c3f-4810-99da-075598b7ff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:08.850784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-30cac039-4c3f-4810-99da-075598b7ff0b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:08.851959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-30cac039-4c3f-4810-99da-075598b7ff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:08.860782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:37:08.898179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebad46d-845a-49f8-a081-bcba9e479e4d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:08.909961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebad46d-845a-49f8-a081-bcba9e479e4d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:08.913789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57765537-aa57-4159-abf3-39ea2eee2a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:08.914421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57765537-aa57-4159-abf3-39ea2eee2a57 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:08.914695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57765537-aa57-4159-abf3-39ea2eee2a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:08.914942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57765537-aa57-4159-abf3-39ea2eee2a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:08.924014 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57765537-aa57-4159-abf3-39ea2eee2a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:08.925835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57765537-aa57-4159-abf3-39ea2eee2a57 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:08.935271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 725/1455] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:37:11.246013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010a00c-f8f2-4f8f-9ab4-b0834bed6453 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:11.246619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010a00c-f8f2-4f8f-9ab4-b0834bed6453 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:11.246619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010a00c-f8f2-4f8f-9ab4-b0834bed6453 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:11.248708 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:37:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:11.256951 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1a4c-8f87-4ac2-97aa-ccb9fc76cac0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:11.257420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1a4c-8f87-4ac2-97aa-ccb9fc76cac0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:11.257533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1a4c-8f87-4ac2-97aa-ccb9fc76cac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:11.278338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b5dd1a4c-8f87-4ac2-97aa-ccb9fc76cac0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:11.279531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b5dd1a4c-8f87-4ac2-97aa-ccb9fc76cac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:11.289345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:37:11.328200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010a00c-f8f2-4f8f-9ab4-b0834bed6453 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:11.340070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010a00c-f8f2-4f8f-9ab4-b0834bed6453 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:11.344148 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b916de-f9eb-4c9d-90ca-159c304cc768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:11.344814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b916de-f9eb-4c9d-90ca-159c304cc768 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:11.345093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b916de-f9eb-4c9d-90ca-159c304cc768 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:11.345394 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b916de-f9eb-4c9d-90ca-159c304cc768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:11.354821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b916de-f9eb-4c9d-90ca-159c304cc768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:11.356264 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b916de-f9eb-4c9d-90ca-159c304cc768 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:11.365693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 732/1458] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:37:13.635462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358110e0-a527-43fc-aa60-f4ff87584f93 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:13.635995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358110e0-a527-43fc-aa60-f4ff87584f93 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:13.635995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358110e0-a527-43fc-aa60-f4ff87584f93 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:13.637427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:13.643806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe3a6d-61db-4793-aacc-6b1eaed7a96a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:13.644404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe3a6d-61db-4793-aacc-6b1eaed7a96a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:13.644526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe3a6d-61db-4793-aacc-6b1eaed7a96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:13.662951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-effe3a6d-61db-4793-aacc-6b1eaed7a96a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:13.664142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-effe3a6d-61db-4793-aacc-6b1eaed7a96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:13.673904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:37:13.711978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358110e0-a527-43fc-aa60-f4ff87584f93 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:13.726714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358110e0-a527-43fc-aa60-f4ff87584f93 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:13.732799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595d799-79a2-4c0e-b01a-f06232a98f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:13.733756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595d799-79a2-4c0e-b01a-f06232a98f3e service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:13.734170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595d799-79a2-4c0e-b01a-f06232a98f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:13.734521 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595d799-79a2-4c0e-b01a-f06232a98f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:13.746204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595d799-79a2-4c0e-b01a-f06232a98f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:13.747630 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595d799-79a2-4c0e-b01a-f06232a98f3e service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:13.757546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 728/1461] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:37:16.040070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99333916-df4a-4957-bde8-3daa49b8ecd6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:16.040807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99333916-df4a-4957-bde8-3daa49b8ecd6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:16.040807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99333916-df4a-4957-bde8-3daa49b8ecd6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:16.042017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:16.048630 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ee401-e4a9-4f9d-a699-f34d4183eaa2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:16.049392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ee401-e4a9-4f9d-a699-f34d4183eaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:16.049524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ee401-e4a9-4f9d-a699-f34d4183eaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:16.069888 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-276ee401-e4a9-4f9d-a699-f34d4183eaa2 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:16.071092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-276ee401-e4a9-4f9d-a699-f34d4183eaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:16.080206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:37:16.120518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99333916-df4a-4957-bde8-3daa49b8ecd6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:16.132989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99333916-df4a-4957-bde8-3daa49b8ecd6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:16.137282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79efae0-22df-489b-83d9-e90f44ac3554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:16.137912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79efae0-22df-489b-83d9-e90f44ac3554 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:16.138186 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79efae0-22df-489b-83d9-e90f44ac3554 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:16.138462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79efae0-22df-489b-83d9-e90f44ac3554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:16.148063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79efae0-22df-489b-83d9-e90f44ac3554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:16.149516 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79efae0-22df-489b-83d9-e90f44ac3554 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:16.158785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 735/1464] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:37:18.450767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2cc5d-b1da-46ae-b1fa-2fba2b33588f None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:18.451384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2cc5d-b1da-46ae-b1fa-2fba2b33588f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:18.451384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2cc5d-b1da-46ae-b1fa-2fba2b33588f None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:18.452710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:18.459130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8d368-4af3-416e-9805-bbef7aba6fab None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:18.460688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8d368-4af3-416e-9805-bbef7aba6fab None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:18.461150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8d368-4af3-416e-9805-bbef7aba6fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:18.484286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-43e8d368-4af3-416e-9805-bbef7aba6fab None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:18.485532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-43e8d368-4af3-416e-9805-bbef7aba6fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:18.497075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:37:18.538470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2cc5d-b1da-46ae-b1fa-2fba2b33588f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:18.552083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2cc5d-b1da-46ae-b1fa-2fba2b33588f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:18.557986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d94f07-75b3-44b4-a079-eec2416e20ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:18.559025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d94f07-75b3-44b4-a079-eec2416e20ef service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:18.559486 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d94f07-75b3-44b4-a079-eec2416e20ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:18.559933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d94f07-75b3-44b4-a079-eec2416e20ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:18.570202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d94f07-75b3-44b4-a079-eec2416e20ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:18.571642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d94f07-75b3-44b4-a079-eec2416e20ef service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:18.580781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 731/1467] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:37:20.826706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac3e0-031c-420a-af77-81483db511e5 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:20.827306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac3e0-031c-420a-af77-81483db511e5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:20.827306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac3e0-031c-420a-af77-81483db511e5 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:20.828655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:20.835576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1309710-ab73-49f5-afe3-48edb70a29e0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:20.836305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1309710-ab73-49f5-afe3-48edb70a29e0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:20.836448 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1309710-ab73-49f5-afe3-48edb70a29e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:20.860310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c1309710-ab73-49f5-afe3-48edb70a29e0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:20.861547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c1309710-ab73-49f5-afe3-48edb70a29e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:20.871209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:37:20.907948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac3e0-031c-420a-af77-81483db511e5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:20.920721 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac3e0-031c-420a-af77-81483db511e5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:20.924712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfb0d6-53dc-4cdd-a78b-65484ba03416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:20.925330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfb0d6-53dc-4cdd-a78b-65484ba03416 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:20.925664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfb0d6-53dc-4cdd-a78b-65484ba03416 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:20.925938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfb0d6-53dc-4cdd-a78b-65484ba03416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:20.935337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfb0d6-53dc-4cdd-a78b-65484ba03416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:20.936819 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfb0d6-53dc-4cdd-a78b-65484ba03416 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:20.946193 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 738/1470] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:37:53.852948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c87519-a655-436d-8e11-d89a047dac0f None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:53.853585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c87519-a655-436d-8e11-d89a047dac0f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:53.853585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c87519-a655-436d-8e11-d89a047dac0f None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:53.854874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 733/1471] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:53.861212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f453-b048-4018-a0f7-886c975bfd13 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:53.861683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f453-b048-4018-a0f7-886c975bfd13 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:53.861794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f453-b048-4018-a0f7-886c975bfd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:53.888159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0d05f453-b048-4018-a0f7-886c975bfd13 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:53.889311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0d05f453-b048-4018-a0f7-886c975bfd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:53.899594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 739/1472] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:37:53.952532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d527e8b-bd74-4306-84b1-9da8063b4b1a None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:53.952798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d527e8b-bd74-4306-84b1-9da8063b4b1a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:53.953105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d527e8b-bd74-4306-84b1-9da8063b4b1a None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:53.954467 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 734/1473] 10.222.0.50 () {54 vars in 962 bytes} [Fri Apr 29 18:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:53.960283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711301a-6d0e-4fea-8598-eacc6a808359 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:53.960608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711301a-6d0e-4fea-8598-eacc6a808359 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:53.960885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711301a-6d0e-4fea-8598-eacc6a808359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:53.979856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8711301a-6d0e-4fea-8598-eacc6a808359 None None] MFA Rules not processed for user `6552c41f1c654ef5bfdd940e3b807dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:53.981096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8711301a-6d0e-4fea-8598-eacc6a808359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:53.988066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 740/1474] 10.222.0.50 () {58 vars in 1059 bytes} [Fri Apr 29 18:37:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:37:53.995988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d527e8b-bd74-4306-84b1-9da8063b4b1a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:54.007165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d527e8b-bd74-4306-84b1-9da8063b4b1a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:54.010833 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8114-e514-4730-bcb8-3a10cbbc397d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6552c41f1c654ef5bfdd940e3b807dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:54.011464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8114-e514-4730-bcb8-3a10cbbc397d service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:54.011741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8114-e514-4730-bcb8-3a10cbbc397d service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:54.011997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8114-e514-4730-bcb8-3a10cbbc397d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:54.021269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c8114-e514-4730-bcb8-3a10cbbc397d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:54.022695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c8114-e514-4730-bcb8-3a10cbbc397d service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:54.031747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 735/1475] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:37:56.357635 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb289d0-1915-4684-ae89-fb02020ebd63 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:56.358510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb289d0-1915-4684-ae89-fb02020ebd63 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:56.358510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb289d0-1915-4684-ae89-fb02020ebd63 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:56.360705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 741/1476] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:37:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:56.367276 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa2c7d-a055-4d3e-a399-a21c635e5b6f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:56.367745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa2c7d-a055-4d3e-a399-a21c635e5b6f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:56.367850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa2c7d-a055-4d3e-a399-a21c635e5b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:56.385767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f7fa2c7d-a055-4d3e-a399-a21c635e5b6f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:56.386930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f7fa2c7d-a055-4d3e-a399-a21c635e5b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:56.395471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 736/1477] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:37:56.430579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb289d0-1915-4684-ae89-fb02020ebd63 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:56.444639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb289d0-1915-4684-ae89-fb02020ebd63 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:56.449059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc09fe6-0440-4933-9149-479a7a3c1f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:56.449711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc09fe6-0440-4933-9149-479a7a3c1f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:56.449993 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc09fe6-0440-4933-9149-479a7a3c1f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:56.450266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc09fe6-0440-4933-9149-479a7a3c1f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:56.451010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc09fe6-0440-4933-9149-479a7a3c1f2c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:56.452821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-7bc09fe6-0440-4933-9149-479a7a3c1f2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:37:56.452821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Apr 29 18:37:56.452821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: Apr 29 18:37:56.453975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc09fe6-0440-4933-9149-479a7a3c1f2c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:56.475504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 742/1478] 10.222.0.50 () {56 vars in 1108 bytes} [Fri Apr 29 18:37:56 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:37:58.747977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb8214-008f-4932-bf64-aafffb9d8c38 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:58.748935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb8214-008f-4932-bf64-aafffb9d8c38 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:58.748935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb8214-008f-4932-bf64-aafffb9d8c38 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:58.751148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 737/1479] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:37:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:37:58.758065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03519-5dfa-46cf-9ee5-f362826d72f6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:58.758812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03519-5dfa-46cf-9ee5-f362826d72f6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:58.758958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03519-5dfa-46cf-9ee5-f362826d72f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:58.787599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-98e03519-5dfa-46cf-9ee5-f362826d72f6 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:37:58.789426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-98e03519-5dfa-46cf-9ee5-f362826d72f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:37:58.802773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 743/1480] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:37:58.850275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb8214-008f-4932-bf64-aafffb9d8c38 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:37:58.875804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb8214-008f-4932-bf64-aafffb9d8c38 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:37:58.884280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc344236-4532-44c5-98d0-be472dbcbeab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:37:58.884944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344236-4532-44c5-98d0-be472dbcbeab service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:37:58.885225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344236-4532-44c5-98d0-be472dbcbeab service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:37:58.885463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344236-4532-44c5-98d0-be472dbcbeab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:37:58.894367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc344236-4532-44c5-98d0-be472dbcbeab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:37:58.896122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc344236-4532-44c5-98d0-be472dbcbeab service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:37:58.905917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 738/1481] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:38:01.348086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060f37a-5728-483b-8279-b9868e1fef7d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:01.349062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060f37a-5728-483b-8279-b9868e1fef7d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:01.349062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060f37a-5728-483b-8279-b9868e1fef7d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:01.350097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 744/1482] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:38:01.356251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ecc98-ff2f-4b6b-a806-78b6557ca028 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:01.356778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ecc98-ff2f-4b6b-a806-78b6557ca028 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:01.356877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ecc98-ff2f-4b6b-a806-78b6557ca028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:01.382899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a89ecc98-ff2f-4b6b-a806-78b6557ca028 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:38:01.384117 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a89ecc98-ff2f-4b6b-a806-78b6557ca028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:38:01.393540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 739/1483] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:38:01.432022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060f37a-5728-483b-8279-b9868e1fef7d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:38:01.449780 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060f37a-5728-483b-8279-b9868e1fef7d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:38:01.456110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f3c5d-2a9a-4bc9-a550-d6d5e69e01c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:38:01.457257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f3c5d-2a9a-4bc9-a550-d6d5e69e01c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:01.457593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f3c5d-2a9a-4bc9-a550-d6d5e69e01c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:01.457948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f3c5d-2a9a-4bc9-a550-d6d5e69e01c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:01.474957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f3c5d-2a9a-4bc9-a550-d6d5e69e01c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:38:01.476402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f3c5d-2a9a-4bc9-a550-d6d5e69e01c9 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:38:01.486897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 745/1484] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:38:03.787836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaea71-5213-47d6-a8b0-7a631ef02046 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:03.788475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaea71-5213-47d6-a8b0-7a631ef02046 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:03.788475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaea71-5213-47d6-a8b0-7a631ef02046 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:03.789750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 740/1485] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:38:03.796180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2dbc-5fd5-4996-af39-be8ddbbddb51 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:03.797080 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2dbc-5fd5-4996-af39-be8ddbbddb51 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:03.797080 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2dbc-5fd5-4996-af39-be8ddbbddb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:03.817555 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-76cc2dbc-5fd5-4996-af39-be8ddbbddb51 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:38:03.818758 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-76cc2dbc-5fd5-4996-af39-be8ddbbddb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:38:03.827857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 746/1486] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:38:03.868106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaea71-5213-47d6-a8b0-7a631ef02046 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:38:03.886150 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaea71-5213-47d6-a8b0-7a631ef02046 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:38:03.893141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94215cf4-4195-4717-a7ff-594cf084eaa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:38:03.894012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94215cf4-4195-4717-a7ff-594cf084eaa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:03.894376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94215cf4-4195-4717-a7ff-594cf084eaa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:03.894725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94215cf4-4195-4717-a7ff-594cf084eaa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:03.905617 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94215cf4-4195-4717-a7ff-594cf084eaa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:38:03.907716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94215cf4-4195-4717-a7ff-594cf084eaa7 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:38:03.920820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 741/1487] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:38:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:38:06.258183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f2441-2406-418d-bae5-14785644ef87 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:06.259082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f2441-2406-418d-bae5-14785644ef87 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:06.259082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f2441-2406-418d-bae5-14785644ef87 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:06.261429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 747/1488] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:38:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:38:06.269181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95c066-23cb-45b6-9c3c-12a9b2925c27 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:06.269640 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95c066-23cb-45b6-9c3c-12a9b2925c27 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:06.269742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95c066-23cb-45b6-9c3c-12a9b2925c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:06.289119 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bc95c066-23cb-45b6-9c3c-12a9b2925c27 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:38:06.290291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bc95c066-23cb-45b6-9c3c-12a9b2925c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:38:06.298906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 742/1489] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:38:06.344298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f2441-2406-418d-bae5-14785644ef87 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:38:06.357147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f2441-2406-418d-bae5-14785644ef87 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:38:06.360849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80426bb1-da61-4f0a-99e8-d45a467baa54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:38:06.361486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80426bb1-da61-4f0a-99e8-d45a467baa54 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:06.361763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80426bb1-da61-4f0a-99e8-d45a467baa54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:06.362030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80426bb1-da61-4f0a-99e8-d45a467baa54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:06.371356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80426bb1-da61-4f0a-99e8-d45a467baa54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:38:06.372815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80426bb1-da61-4f0a-99e8-d45a467baa54 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:38:06.382175 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 748/1490] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:38:08.828629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004462ac-4221-4455-b279-c7d612e79d4d None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:08.829249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004462ac-4221-4455-b279-c7d612e79d4d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:08.829249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004462ac-4221-4455-b279-c7d612e79d4d None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:08.830519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 743/1491] 10.222.0.50 () {54 vars in 926 bytes} [Fri Apr 29 18:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:38:08.836607 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c7239-8122-454c-93db-a7fbbfc0e7c3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:08.837091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c7239-8122-454c-93db-a7fbbfc0e7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:08.837201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c7239-8122-454c-93db-a7fbbfc0e7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:08.857941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-605c7239-8122-454c-93db-a7fbbfc0e7c3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:38:08.859149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-605c7239-8122-454c-93db-a7fbbfc0e7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:38:08.868717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 749/1492] 10.222.0.50 () {58 vars in 1023 bytes} [Fri Apr 29 18:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:38:08.910559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004462ac-4221-4455-b279-c7d612e79d4d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:38:08.923632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004462ac-4221-4455-b279-c7d612e79d4d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:38:08.928615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1fe55-4106-402a-8056-56eea30f0e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:38:08.929242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1fe55-4106-402a-8056-56eea30f0e6d service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:38:08.929516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1fe55-4106-402a-8056-56eea30f0e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:38:08.929779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1fe55-4106-402a-8056-56eea30f0e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:38:08.939437 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1fe55-4106-402a-8056-56eea30f0e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:38:08.940900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1fe55-4106-402a-8056-56eea30f0e6d service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:38:08.953334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 744/1493] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:39:36.586643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2f631-c0cf-4790-b427-d994a4cf5065 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:36.587287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2f631-c0cf-4790-b427-d994a4cf5065 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:36.587287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2f631-c0cf-4790-b427-d994a4cf5065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:36.607111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a8b2f631-c0cf-4790-b427-d994a4cf5065 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:39:36.608321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a8b2f631-c0cf-4790-b427-d994a4cf5065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:39:36.619619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 750/1494] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:39:36.628906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62f956-7089-4b2d-86cc-0d5125a12247 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:36.629373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62f956-7089-4b2d-86cc-0d5125a12247 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:36.629494 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62f956-7089-4b2d-86cc-0d5125a12247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:36.647780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1e62f956-7089-4b2d-86cc-0d5125a12247 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:39:36.649016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1e62f956-7089-4b2d-86cc-0d5125a12247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:39:36.657347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 745/1495] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:39:36.662698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2f631-c0cf-4790-b427-d994a4cf5065 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:39:36.678819 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2f631-c0cf-4790-b427-d994a4cf5065 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:39:36.686800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a47603-cd18-4743-9d30-61a47299ad90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:39:36.687766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a47603-cd18-4743-9d30-61a47299ad90 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:36.688136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a47603-cd18-4743-9d30-61a47299ad90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:36.688403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a47603-cd18-4743-9d30-61a47299ad90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:36.689500 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a47603-cd18-4743-9d30-61a47299ad90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:39:36.689871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a47603-cd18-4743-9d30-61a47299ad90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:39:36.691088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-f1a47603-cd18-4743-9d30-61a47299ad90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:39:36.691088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Apr 29 18:39:36.691088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: Apr 29 18:39:36.691978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a47603-cd18-4743-9d30-61a47299ad90 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:39:36.703756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 751/1496] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:39:36.712189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62f956-7089-4b2d-86cc-0d5125a12247 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:39:36.721849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62f956-7089-4b2d-86cc-0d5125a12247 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:39:36.727680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456b43f-d1e1-4f5a-b4e5-c591052baf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:39:36.728314 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456b43f-d1e1-4f5a-b4e5-c591052baf04 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:36.728659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456b43f-d1e1-4f5a-b4e5-c591052baf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:36.728926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456b43f-d1e1-4f5a-b4e5-c591052baf04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:36.729617 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456b43f-d1e1-4f5a-b4e5-c591052baf04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:39:36.731425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-5456b43f-d1e1-4f5a-b4e5-c591052baf04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:39:36.731425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Apr 29 18:39:36.731425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: Apr 29 18:39:36.732661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456b43f-d1e1-4f5a-b4e5-c591052baf04 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:39:36.773837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 746/1497] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:39:36 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:39:36.778211 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a47603-cd18-4743-9d30-61a47299ad90 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:39:36.788282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a47603-cd18-4743-9d30-61a47299ad90 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:39:36.846498 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58ada-9460-49de-8acc-579420d2781a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:39:36.847101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58ada-9460-49de-8acc-579420d2781a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:36.847403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58ada-9460-49de-8acc-579420d2781a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:36.847673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58ada-9460-49de-8acc-579420d2781a admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:36.848364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee58ada-9460-49de-8acc-579420d2781a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:39:36.849421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-fee58ada-9460-49de-8acc-579420d2781a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:39:36.849421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Apr 29 18:39:36.849421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: Apr 29 18:39:36.850611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee58ada-9460-49de-8acc-579420d2781a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:39:36.876241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 752/1498] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:39:36 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:39:36.882279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456b43f-d1e1-4f5a-b4e5-c591052baf04 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:39:36.893832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456b43f-d1e1-4f5a-b4e5-c591052baf04 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:39:36.899884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97500f6e-4c06-4228-a50c-ab44cf377bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:39:36.900538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97500f6e-4c06-4228-a50c-ab44cf377bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:36.900848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97500f6e-4c06-4228-a50c-ab44cf377bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:36.901118 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97500f6e-4c06-4228-a50c-ab44cf377bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:36.901846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97500f6e-4c06-4228-a50c-ab44cf377bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:39:36.902874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-97500f6e-4c06-4228-a50c-ab44cf377bf8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:39:36.902874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Apr 29 18:39:36.902874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: Apr 29 18:39:36.903787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97500f6e-4c06-4228-a50c-ab44cf377bf8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:39:36.917001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 747/1499] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:39:36.921040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58ada-9460-49de-8acc-579420d2781a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:39:36.931315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58ada-9460-49de-8acc-579420d2781a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:39:36.937021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5914c0-8d53-4171-aefc-e705a8469bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:39:36.937673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5914c0-8d53-4171-aefc-e705a8469bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:36.937951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5914c0-8d53-4171-aefc-e705a8469bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:36.938214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5914c0-8d53-4171-aefc-e705a8469bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:36.938936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5914c0-8d53-4171-aefc-e705a8469bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:39:36.939955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-6c5914c0-8d53-4171-aefc-e705a8469bf5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:39:36.939955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Apr 29 18:39:36.939955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: Apr 29 18:39:36.940892 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5914c0-8d53-4171-aefc-e705a8469bf5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:39:36.954288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 753/1500] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:39:36.959069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97500f6e-4c06-4228-a50c-ab44cf377bf8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:39:36.970126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97500f6e-4c06-4228-a50c-ab44cf377bf8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:39:36.976518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe683a-4af7-4bc3-8e88-740033d92b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:39:36.977446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe683a-4af7-4bc3-8e88-740033d92b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:36.977724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe683a-4af7-4bc3-8e88-740033d92b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:36.977989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe683a-4af7-4bc3-8e88-740033d92b58 admin admin] PATH_INFO: `/v3/projects/15e4773ea7ed4899b0dc0e15abef073e/users/5da75998dfef4bc9a67af93f18fd1952/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:36.994299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe683a-4af7-4bc3-8e88-740033d92b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e4773ea7ed4899b0dc0e15abef073e, user_id=5da75998dfef4bc9a67af93f18fd1952, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:39:36.995346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-42fe683a-4af7-4bc3-8e88-740033d92b58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:39:36.995346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Apr 29 18:39:36.995346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: Apr 29 18:39:36.997332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe683a-4af7-4bc3-8e88-740033d92b58 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:39:37.013466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 748/1501] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:39:36 2022] PUT /identity/v3/projects/15e4773ea7ed4899b0dc0e15abef073e/users/5da75998dfef4bc9a67af93f18fd1952/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:39:37.025058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ad67b-5d32-4423-9e24-8fca57d51121 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:37.025425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ad67b-5d32-4423-9e24-8fca57d51121 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:37.025704 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ad67b-5d32-4423-9e24-8fca57d51121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:37.053159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f10ad67b-5d32-4423-9e24-8fca57d51121 None None] MFA Rules not processed for user `5da75998dfef4bc9a67af93f18fd1952`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:39:37.054351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f10ad67b-5d32-4423-9e24-8fca57d51121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:39:37.122739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 754/1502] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:39:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:39:37.134471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe683a-4af7-4bc3-8e88-740033d92b58 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:39:37.206524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe683a-4af7-4bc3-8e88-740033d92b58 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:39:37.213682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f101e-1140-4de6-b70d-97aa611c90d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:39:37.214327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f101e-1140-4de6-b70d-97aa611c90d4 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:37.214607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f101e-1140-4de6-b70d-97aa611c90d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:37.214875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f101e-1140-4de6-b70d-97aa611c90d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:37.223552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f101e-1140-4de6-b70d-97aa611c90d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:39:37.225191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f101e-1140-4de6-b70d-97aa611c90d4 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:39:37.234342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 749/1503] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:39:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:39:37.314840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ab3c-c4f8-40d0-9bee-a5170143d569 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:37.315099 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ab3c-c4f8-40d0-9bee-a5170143d569 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:37.315404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ab3c-c4f8-40d0-9bee-a5170143d569 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:37.319415 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 755/1504] 10.222.0.50 () {52 vars in 829 bytes} [Fri Apr 29 18:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:39:37.320722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f101e-1140-4de6-b70d-97aa611c90d4 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:39:37.382308 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f101e-1140-4de6-b70d-97aa611c90d4 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:39:37.387470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:39:37.388338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:37.388652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:37.388932 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] PATH_INFO: `/v3/users/5da75998dfef4bc9a67af93f18fd1952` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:37.390165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da75998dfef4bc9a67af93f18fd1952)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:39:37.391191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:39:37.391191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Apr 29 18:39:37.391191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: Apr 29 18:39:37.392393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:39:37.460661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da75998dfef4bc9a67af93f18fd1952'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:39:37.461312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da75998dfef4bc9a67af93f18fd1952'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:39:37.479530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da75998dfef4bc9a67af93f18fd1952'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:39:37.493719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da75998dfef4bc9a67af93f18fd1952'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:39:37.507717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da75998dfef4bc9a67af93f18fd1952'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:39:37.515605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 750/1505] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:39:37 2022] DELETE /identity/v3/users/5da75998dfef4bc9a67af93f18fd1952 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:39:37.520765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193095c-df59-4bed-bda8-874dc1ca56ea None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:37.521053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193095c-df59-4bed-bda8-874dc1ca56ea None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:37.521329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193095c-df59-4bed-bda8-874dc1ca56ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:37.539160 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c193095c-df59-4bed-bda8-874dc1ca56ea None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:39:37.540364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c193095c-df59-4bed-bda8-874dc1ca56ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:39:37.603837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 756/1506] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:39:37.613736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:39:37.615359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:39:37.677300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204ae24-6456-4499-92c3-49adb694561b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:39:37.681541 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba89eca-1894-49ef-8bae-442f7d4f0b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:39:37.682205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89eca-1894-49ef-8bae-442f7d4f0b80 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:37.682459 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89eca-1894-49ef-8bae-442f7d4f0b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:37.682726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89eca-1894-49ef-8bae-442f7d4f0b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:37.693790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba89eca-1894-49ef-8bae-442f7d4f0b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:39:37.695312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba89eca-1894-49ef-8bae-442f7d4f0b80 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:39:37.708096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 751/1507] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:39:38.061080 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193095c-df59-4bed-bda8-874dc1ca56ea None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:39:38.062651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c193095c-df59-4bed-bda8-874dc1ca56ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:39:38.075204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193095c-df59-4bed-bda8-874dc1ca56ea None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:39:38.080347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:39:38.081362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:39:38.081648 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:39:38.081923 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] PATH_INFO: `/v3/projects/15e4773ea7ed4899b0dc0e15abef073e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:39:38.083642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e4773ea7ed4899b0dc0e15abef073e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:39:38.084723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:39:38.084723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Apr 29 18:39:38.084723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: Apr 29 18:39:38.085941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:39:38.150064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e4773ea7ed4899b0dc0e15abef073e', 'request_id': 'req-99458fac-1398-466b-a34c-5d51b506f25c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:39:38.161066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e4773ea7ed4899b0dc0e15abef073e', 'request_id': 'req-99458fac-1398-466b-a34c-5d51b506f25c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:39:38.174588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e4773ea7ed4899b0dc0e15abef073e', 'request_id': 'req-99458fac-1398-466b-a34c-5d51b506f25c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:39:38.186011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] The token cache is being invalidate because project 15e4773ea7ed4899b0dc0e15abef073e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:39:38.186202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:39:38.186890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99458fac-1398-466b-a34c-5d51b506f25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:39:38.188579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 757/1508] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:39:38 2022] DELETE /identity/v3/projects/15e4773ea7ed4899b0dc0e15abef073e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:41:17.064641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d2796-aeb0-4747-8452-9a20ef26b303 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:17.065806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d2796-aeb0-4747-8452-9a20ef26b303 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:17.065806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d2796-aeb0-4747-8452-9a20ef26b303 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:17.067138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 752/1509] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:41:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:41:17.073307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad6153-ffaa-4e93-9385-48d08594527e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:17.073998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad6153-ffaa-4e93-9385-48d08594527e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:17.074156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad6153-ffaa-4e93-9385-48d08594527e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:17.102678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ecad6153-ffaa-4e93-9385-48d08594527e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:17.103910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ecad6153-ffaa-4e93-9385-48d08594527e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:17.175082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 758/1510] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:41:17.333027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d2796-aeb0-4747-8452-9a20ef26b303 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:41:17.335249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-667d2796-aeb0-4747-8452-9a20ef26b303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:17.414586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d2796-aeb0-4747-8452-9a20ef26b303 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:41:17.420796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b98b43-58f6-4be9-92d2-bbbfa2f7266e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:41:17.421445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b98b43-58f6-4be9-92d2-bbbfa2f7266e service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:17.421727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b98b43-58f6-4be9-92d2-bbbfa2f7266e service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:17.422001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b98b43-58f6-4be9-92d2-bbbfa2f7266e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:17.431076 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b98b43-58f6-4be9-92d2-bbbfa2f7266e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:41:17.433628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b98b43-58f6-4be9-92d2-bbbfa2f7266e service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:41:17.449112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 753/1511] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:41:18.322603 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963956b8-6e38-4b8e-905e-69c1fb831c6a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:18.323159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963956b8-6e38-4b8e-905e-69c1fb831c6a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:18.323159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963956b8-6e38-4b8e-905e-69c1fb831c6a None None] PATH_INFO: `/v3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:18.324376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 759/1512] 10.222.0.15 () {54 vars in 925 bytes} [Fri Apr 29 18:41:18 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:41:18.332284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66db2f-0ce9-4ff4-a739-f4d906efc953 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:18.332798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66db2f-0ce9-4ff4-a739-f4d906efc953 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:18.332894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66db2f-0ce9-4ff4-a739-f4d906efc953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:18.352507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5c66db2f-0ce9-4ff4-a739-f4d906efc953 None None] MFA Rules not processed for user `6e934c8e7dd0425cac68eb91e5e2c0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:18.353734 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5c66db2f-0ce9-4ff4-a739-f4d906efc953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:18.420871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 754/1513] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Apr 29 18:41:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:41:18.658302 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed73e04-f0c6-4211-bc63-1c9e7b0fa261 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:18.658590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed73e04-f0c6-4211-bc63-1c9e7b0fa261 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:18.658864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed73e04-f0c6-4211-bc63-1c9e7b0fa261 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:18.660416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:41:18.665426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ef3e2-e41f-4dd9-b3bd-5fc371d72b08 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:18.665714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ef3e2-e41f-4dd9-b3bd-5fc371d72b08 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:18.665988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ef3e2-e41f-4dd9-b3bd-5fc371d72b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:18.684992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d96ef3e2-e41f-4dd9-b3bd-5fc371d72b08 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:18.686183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d96ef3e2-e41f-4dd9-b3bd-5fc371d72b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:18.694892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:41:18.847859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed73e04-f0c6-4211-bc63-1c9e7b0fa261 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:41:18.869030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed73e04-f0c6-4211-bc63-1c9e7b0fa261 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:41:18.875836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf72548-9e9b-4831-9cd2-121e9919a284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:41:18.876426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf72548-9e9b-4831-9cd2-121e9919a284 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:18.876732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf72548-9e9b-4831-9cd2-121e9919a284 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:18.877005 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf72548-9e9b-4831-9cd2-121e9919a284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:18.885854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf72548-9e9b-4831-9cd2-121e9919a284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:41:18.887489 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf72548-9e9b-4831-9cd2-121e9919a284 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:41:18.897234 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 761/1516] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:41:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:41:27.754801 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d233-0ee3-4aca-87c7-8517b0c3dc0d None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:27.756121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d233-0ee3-4aca-87c7-8517b0c3dc0d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:27.756121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d233-0ee3-4aca-87c7-8517b0c3dc0d None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:27.757103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 756/1517] 10.222.0.50 () {54 vars in 960 bytes} [Fri Apr 29 18:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:41:27.762232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57ed1-880d-4a42-8e0e-68739abe898b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:27.762677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57ed1-880d-4a42-8e0e-68739abe898b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:27.762773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb57ed1-880d-4a42-8e0e-68739abe898b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:27.782672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-beb57ed1-880d-4a42-8e0e-68739abe898b None None] MFA Rules not processed for user `6e934c8e7dd0425cac68eb91e5e2c0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:27.783885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-beb57ed1-880d-4a42-8e0e-68739abe898b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:27.798641 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 762/1518] 10.222.0.50 () {58 vars in 1057 bytes} [Fri Apr 29 18:41:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:41:27.809831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d233-0ee3-4aca-87c7-8517b0c3dc0d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:41:27.825622 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d233-0ee3-4aca-87c7-8517b0c3dc0d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:41:27.832232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917e8da-17ee-47c5-b566-8851aa590fe0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e934c8e7dd0425cac68eb91e5e2c0ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:41:27.832913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917e8da-17ee-47c5-b566-8851aa590fe0 service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:27.833250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917e8da-17ee-47c5-b566-8851aa590fe0 service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:27.833528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917e8da-17ee-47c5-b566-8851aa590fe0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:27.842074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917e8da-17ee-47c5-b566-8851aa590fe0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:41:27.843673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917e8da-17ee-47c5-b566-8851aa590fe0 service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:41:27.853298 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 757/1519] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:41:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:41:30.564425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a0462-eb08-4185-8078-656aeae5037b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:30.565069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a0462-eb08-4185-8078-656aeae5037b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:30.565069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a0462-eb08-4185-8078-656aeae5037b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:30.567032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 763/1520] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:41:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:41:30.572610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59a19b-cb98-4769-bf81-9e363a351ae9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:30.573040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59a19b-cb98-4769-bf81-9e363a351ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:30.573174 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59a19b-cb98-4769-bf81-9e363a351ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:30.592637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3c59a19b-cb98-4769-bf81-9e363a351ae9 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:30.593848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3c59a19b-cb98-4769-bf81-9e363a351ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:30.602710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 758/1521] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:41:30.768617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a0462-eb08-4185-8078-656aeae5037b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:41:30.786373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a0462-eb08-4185-8078-656aeae5037b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:41:30.793681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71531bf2-5529-402b-bb0f-b784443e230f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:41:30.794318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71531bf2-5529-402b-bb0f-b784443e230f service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:30.794606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71531bf2-5529-402b-bb0f-b784443e230f service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:30.794877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71531bf2-5529-402b-bb0f-b784443e230f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:30.803594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71531bf2-5529-402b-bb0f-b784443e230f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:41:30.805264 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71531bf2-5529-402b-bb0f-b784443e230f service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:41:30.815221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 764/1522] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:41:32.037930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8176d2-5e31-4cd3-8c6c-cde22f939930 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:32.038507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8176d2-5e31-4cd3-8c6c-cde22f939930 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:32.038507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8176d2-5e31-4cd3-8c6c-cde22f939930 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:32.039850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 759/1523] 127.0.0.1 () {54 vars in 901 bytes} [Fri Apr 29 18:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:41:32.045971 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3841a4-2eaa-4202-a935-d4a543eea01f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:32.046686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3841a4-2eaa-4202-a935-d4a543eea01f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:32.046878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3841a4-2eaa-4202-a935-d4a543eea01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:32.074617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3f3841a4-2eaa-4202-a935-d4a543eea01f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:32.075826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3f3841a4-2eaa-4202-a935-d4a543eea01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:32.086134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 765/1524] 127.0.0.1 () {58 vars in 998 bytes} [Fri Apr 29 18:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:41:32.238931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8176d2-5e31-4cd3-8c6c-cde22f939930 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:41:32.256742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8176d2-5e31-4cd3-8c6c-cde22f939930 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:41:32.262566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaa7b7-14d3-4559-8259-aafec72eb982 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:41:32.263199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaa7b7-14d3-4559-8259-aafec72eb982 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:32.263474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaa7b7-14d3-4559-8259-aafec72eb982 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:32.263745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaa7b7-14d3-4559-8259-aafec72eb982 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:32.272604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaa7b7-14d3-4559-8259-aafec72eb982 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:41:32.274214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaa7b7-14d3-4559-8259-aafec72eb982 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:41:32.283517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 760/1525] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:41:36.618809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1e310-4d86-4300-8fb7-d25f858c0087 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:36.619381 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1e310-4d86-4300-8fb7-d25f858c0087 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:36.619381 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1e310-4d86-4300-8fb7-d25f858c0087 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:36.621032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 766/1526] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:41:36.627013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57f9fc-5c52-4384-b7bb-e520d535f4b3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:36.627738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57f9fc-5c52-4384-b7bb-e520d535f4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:36.627916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57f9fc-5c52-4384-b7bb-e520d535f4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:36.647853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bb57f9fc-5c52-4384-b7bb-e520d535f4b3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:36.649088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bb57f9fc-5c52-4384-b7bb-e520d535f4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:36.657581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 761/1527] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:41:36.694055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1e310-4d86-4300-8fb7-d25f858c0087 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:41:36.696061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-18b1e310-4d86-4300-8fb7-d25f858c0087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:36.756086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1e310-4d86-4300-8fb7-d25f858c0087 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:41:36.760220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bdfd56-62de-434f-be06-e5350d7c4a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:41:36.760882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdfd56-62de-434f-be06-e5350d7c4a46 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:36.761160 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdfd56-62de-434f-be06-e5350d7c4a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:36.761427 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdfd56-62de-434f-be06-e5350d7c4a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:36.771734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bdfd56-62de-434f-be06-e5350d7c4a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:41:36.773198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bdfd56-62de-434f-be06-e5350d7c4a46 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:41:36.783111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 767/1528] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:41:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:41:38.999331 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c8c00-7a40-4b3c-a091-d6170df0520f None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:38.999972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c8c00-7a40-4b3c-a091-d6170df0520f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:38.999972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c8c00-7a40-4b3c-a091-d6170df0520f None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:39.001270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 762/1529] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:41:39.007816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6630fad-ea1c-4f51-bfad-fe6c70327e06 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:39.008623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6630fad-ea1c-4f51-bfad-fe6c70327e06 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:39.008820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6630fad-ea1c-4f51-bfad-fe6c70327e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:39.033008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a6630fad-ea1c-4f51-bfad-fe6c70327e06 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:39.034219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a6630fad-ea1c-4f51-bfad-fe6c70327e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:39.043319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 768/1530] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:41:39.081727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c8c00-7a40-4b3c-a091-d6170df0520f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:41:39.084138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-649c8c00-7a40-4b3c-a091-d6170df0520f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:39.098721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c8c00-7a40-4b3c-a091-d6170df0520f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:41:39.103163 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15a49b-806a-4650-9c22-67778b52c847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:41:39.103835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15a49b-806a-4650-9c22-67778b52c847 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:39.104116 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15a49b-806a-4650-9c22-67778b52c847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:39.104392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15a49b-806a-4650-9c22-67778b52c847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:39.113940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15a49b-806a-4650-9c22-67778b52c847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:41:39.115577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15a49b-806a-4650-9c22-67778b52c847 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:41:39.125528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 763/1531] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:41:42.928765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a59cd-fd98-4a1a-85cf-c4a1a00dd639 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:42.929628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a59cd-fd98-4a1a-85cf-c4a1a00dd639 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:42.929628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a59cd-fd98-4a1a-85cf-c4a1a00dd639 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:42.931938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 769/1532] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:41:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:41:42.937550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e277e-ff88-41c9-8c34-3f5449015e7e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:42.937980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e277e-ff88-41c9-8c34-3f5449015e7e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:42.938105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e277e-ff88-41c9-8c34-3f5449015e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:42.958379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-cd6e277e-ff88-41c9-8c34-3f5449015e7e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:42.959669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cd6e277e-ff88-41c9-8c34-3f5449015e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:42.973483 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 764/1533] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:41:43.246259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a59cd-fd98-4a1a-85cf-c4a1a00dd639 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:41:43.248597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a3a59cd-fd98-4a1a-85cf-c4a1a00dd639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:43.309843 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a59cd-fd98-4a1a-85cf-c4a1a00dd639 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:41:43.314319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e8dc7-47d5-4190-aba7-694c78efbaa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:41:43.314956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e8dc7-47d5-4190-aba7-694c78efbaa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:43.315250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e8dc7-47d5-4190-aba7-694c78efbaa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:43.315525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e8dc7-47d5-4190-aba7-694c78efbaa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:43.325188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e8dc7-47d5-4190-aba7-694c78efbaa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:41:43.326607 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e8dc7-47d5-4190-aba7-694c78efbaa7 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:41:43.336589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 770/1534] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:41:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:41:45.629238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f257c9e-835d-4ab6-8e5a-699031c952fc None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:45.630004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f257c9e-835d-4ab6-8e5a-699031c952fc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:45.630174 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f257c9e-835d-4ab6-8e5a-699031c952fc None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:45.632949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 765/1535] 127.0.0.1 () {54 vars in 916 bytes} [Fri Apr 29 18:41:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:41:45.640464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf495d5e-9db2-422d-b8b0-85305d0305cd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:45.640942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf495d5e-9db2-422d-b8b0-85305d0305cd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:45.641056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf495d5e-9db2-422d-b8b0-85305d0305cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:45.661620 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cf495d5e-9db2-422d-b8b0-85305d0305cd None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:45.662835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cf495d5e-9db2-422d-b8b0-85305d0305cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:45.671568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 771/1536] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Apr 29 18:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:41:45.944400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb1f2-79cc-4a08-b932-f8a71fc9a67b None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:45.944737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb1f2-79cc-4a08-b932-f8a71fc9a67b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:45.945011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb1f2-79cc-4a08-b932-f8a71fc9a67b None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:45.946397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 766/1537] 10.222.0.50 () {54 vars in 962 bytes} [Fri Apr 29 18:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:41:45.952522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810ec40-338d-4284-9047-2b71a9b4ad70 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:45.952867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810ec40-338d-4284-9047-2b71a9b4ad70 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:45.953231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810ec40-338d-4284-9047-2b71a9b4ad70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:45.980307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1810ec40-338d-4284-9047-2b71a9b4ad70 None None] MFA Rules not processed for user `ae4fe640bb564a8abe872cb1ec74dc7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:41:45.981536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1810ec40-338d-4284-9047-2b71a9b4ad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:41:45.989265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 772/1538] 10.222.0.50 () {58 vars in 1059 bytes} [Fri Apr 29 18:41:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:41:45.996841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cb1f2-79cc-4a08-b932-f8a71fc9a67b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:41:46.009016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cb1f2-79cc-4a08-b932-f8a71fc9a67b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:41:46.013496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874da19-d8d3-4f7e-b880-9157dcc9c093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:41:46.014257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874da19-d8d3-4f7e-b880-9157dcc9c093 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:41:46.014520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874da19-d8d3-4f7e-b880-9157dcc9c093 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:41:46.014881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874da19-d8d3-4f7e-b880-9157dcc9c093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:41:46.024303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874da19-d8d3-4f7e-b880-9157dcc9c093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:41:46.025808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874da19-d8d3-4f7e-b880-9157dcc9c093 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:41:46.035197 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 767/1539] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:17.749097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a31811-24ac-4b22-9a6f-cb3c17cf55a1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:17.749097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a31811-24ac-4b22-9a6f-cb3c17cf55a1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:17.749097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a31811-24ac-4b22-9a6f-cb3c17cf55a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:17.773262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-09a31811-24ac-4b22-9a6f-cb3c17cf55a1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:17.774501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-09a31811-24ac-4b22-9a6f-cb3c17cf55a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:17.785872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 773/1540] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:43:17.797379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acbbc7-a7ba-4922-abad-6dd63e034db9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:17.798515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acbbc7-a7ba-4922-abad-6dd63e034db9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:17.799024 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acbbc7-a7ba-4922-abad-6dd63e034db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:17.827242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-21acbbc7-a7ba-4922-abad-6dd63e034db9 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:17.830634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a64df2-2112-447d-9312-ebd935496a56 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:17.830925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a64df2-2112-447d-9312-ebd935496a56 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:17.831205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a64df2-2112-447d-9312-ebd935496a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:17.831882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-21acbbc7-a7ba-4922-abad-6dd63e034db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:17.849949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 768/1541] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:17.851858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623f1c2-0343-4e4b-bf31-bad7cad99b75 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:17.852367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623f1c2-0343-4e4b-bf31-bad7cad99b75 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:17.852877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623f1c2-0343-4e4b-bf31-bad7cad99b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:17.861415 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e7a64df2-2112-447d-9312-ebd935496a56 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:17.862678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e7a64df2-2112-447d-9312-ebd935496a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:17.881794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 774/1542] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:17.882007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a64df2-2112-447d-9312-ebd935496a56 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:17.892714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8623f1c2-0343-4e4b-bf31-bad7cad99b75 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:17.892714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8623f1c2-0343-4e4b-bf31-bad7cad99b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:17.899076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a64df2-2112-447d-9312-ebd935496a56 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:17.908647 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8169d-4491-4a80-9b43-eaa6dea72a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:17.911163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8169d-4491-4a80-9b43-eaa6dea72a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:17.911440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8169d-4491-4a80-9b43-eaa6dea72a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:17.911637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8169d-4491-4a80-9b43-eaa6dea72a18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:17.912412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8169d-4491-4a80-9b43-eaa6dea72a18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:17.913653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8169d-4491-4a80-9b43-eaa6dea72a18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:17.914419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8169d-4491-4a80-9b43-eaa6dea72a18 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:17.916369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 769/1543] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:17.918111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6155f59-11d0-404c-b685-3f0a2799f063 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:17.918403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6155f59-11d0-404c-b685-3f0a2799f063 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:17.918684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6155f59-11d0-404c-b685-3f0a2799f063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:17.937769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 775/1544] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:17.938769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ff5a6-44ca-4a31-99c2-99f433e9b254 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:17.939104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ff5a6-44ca-4a31-99c2-99f433e9b254 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:17.939383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ff5a6-44ca-4a31-99c2-99f433e9b254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:17.950372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c6155f59-11d0-404c-b685-3f0a2799f063 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:17.951643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c6155f59-11d0-404c-b685-3f0a2799f063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:17.970766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 770/1545] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:17.972114 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1a853-3853-47c0-913d-4c4d2f516729 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:17.972397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1a853-3853-47c0-913d-4c4d2f516729 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:17.975868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1a853-3853-47c0-913d-4c4d2f516729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:17.978813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fe1ff5a6-44ca-4a31-99c2-99f433e9b254 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:17.980261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fe1ff5a6-44ca-4a31-99c2-99f433e9b254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:17.989595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 776/1546] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:17.990899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970677a-b201-437c-b100-18cb82423066 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:17.991184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970677a-b201-437c-b100-18cb82423066 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:17.996515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970677a-b201-437c-b100-18cb82423066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.005077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-11f1a853-3853-47c0-913d-4c4d2f516729 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:18.006284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-11f1a853-3853-47c0-913d-4c4d2f516729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:18.014583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 771/1547] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:18.015216 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1a853-3853-47c0-913d-4c4d2f516729 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.022754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6970677a-b201-437c-b100-18cb82423066 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:18.024575 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6970677a-b201-437c-b100-18cb82423066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:18.025632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1a853-3853-47c0-913d-4c4d2f516729 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.030756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2477c2-6dd8-4699-ab94-23f693365a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.031511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2477c2-6dd8-4699-ab94-23f693365a6c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.031761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2477c2-6dd8-4699-ab94-23f693365a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.032027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2477c2-6dd8-4699-ab94-23f693365a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.032748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2477c2-6dd8-4699-ab94-23f693365a6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.034659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2477c2-6dd8-4699-ab94-23f693365a6c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.035447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 777/1548] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:18.037295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa391e-eccc-418b-adcd-2cfc45439e23 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.037810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa391e-eccc-418b-adcd-2cfc45439e23 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.038514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa391e-eccc-418b-adcd-2cfc45439e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.058542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d2aa391e-eccc-418b-adcd-2cfc45439e23 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:18.059728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d2aa391e-eccc-418b-adcd-2cfc45439e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:18.076521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 778/1549] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:18.076521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30657-faaf-4486-b877-a5934a19a8e9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.076521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30657-faaf-4486-b877-a5934a19a8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.076521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30657-faaf-4486-b877-a5934a19a8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.080732 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 772/1550] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.083297 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff84b50-59f2-433d-bc2c-4e8539ac29ae None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.083746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff84b50-59f2-433d-bc2c-4e8539ac29ae None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.084107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff84b50-59f2-433d-bc2c-4e8539ac29ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.088547 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-16e30657-faaf-4486-b877-a5934a19a8e9 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:18.089865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-16e30657-faaf-4486-b877-a5934a19a8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:18.105647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-fff84b50-59f2-433d-bc2c-4e8539ac29ae None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:18.106918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fff84b50-59f2-433d-bc2c-4e8539ac29ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:18.150927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 779/1551] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:18.151850 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30657-faaf-4486-b877-a5934a19a8e9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.162759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 773/1552] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:18.163586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff84b50-59f2-433d-bc2c-4e8539ac29ae None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.163700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30657-faaf-4486-b877-a5934a19a8e9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.168272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65225e1-a1a0-4eaa-a285-de02a658525d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.168968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65225e1-a1a0-4eaa-a285-de02a658525d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.169256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65225e1-a1a0-4eaa-a285-de02a658525d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.169528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65225e1-a1a0-4eaa-a285-de02a658525d admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.170238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65225e1-a1a0-4eaa-a285-de02a658525d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:18.170546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65225e1-a1a0-4eaa-a285-de02a658525d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.172428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff84b50-59f2-433d-bc2c-4e8539ac29ae None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.172850 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65225e1-a1a0-4eaa-a285-de02a658525d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.177171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc655d-bf13-4340-81cf-6325edb7aed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.177813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc655d-bf13-4340-81cf-6325edb7aed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.178089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc655d-bf13-4340-81cf-6325edb7aed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.178357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc655d-bf13-4340-81cf-6325edb7aed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.179071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc655d-bf13-4340-81cf-6325edb7aed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:18.179372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc655d-bf13-4340-81cf-6325edb7aed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.180532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-3ccc655d-bf13-4340-81cf-6325edb7aed6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:43:18.180532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Apr 29 18:43:18.180532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: Apr 29 18:43:18.181421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc655d-bf13-4340-81cf-6325edb7aed6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.184279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 780/1553] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:18.185652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d508a1-2ee7-4a5a-80c2-8278e28466cf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.186008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d508a1-2ee7-4a5a-80c2-8278e28466cf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.186346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d508a1-2ee7-4a5a-80c2-8278e28466cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.193005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 774/1554] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:18.194326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cafe0-f3f9-45ce-afce-0fdaee3cda5b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.194613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cafe0-f3f9-45ce-afce-0fdaee3cda5b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.194884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cafe0-f3f9-45ce-afce-0fdaee3cda5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.206194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d8d508a1-2ee7-4a5a-80c2-8278e28466cf None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:18.207386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d8d508a1-2ee7-4a5a-80c2-8278e28466cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:18.212526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6b6cafe0-f3f9-45ce-afce-0fdaee3cda5b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:18.213722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6b6cafe0-f3f9-45ce-afce-0fdaee3cda5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:18.215181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 781/1555] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:18.215912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d508a1-2ee7-4a5a-80c2-8278e28466cf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.221838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 775/1556] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:18.225597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00859d-193f-4dd4-8d22-218614a15a79 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.225883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00859d-193f-4dd4-8d22-218614a15a79 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.226157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00859d-193f-4dd4-8d22-218614a15a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.226295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d508a1-2ee7-4a5a-80c2-8278e28466cf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.230749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63effe0f-53fb-4144-b520-9648763205ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.231411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63effe0f-53fb-4144-b520-9648763205ae admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.231690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63effe0f-53fb-4144-b520-9648763205ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.231964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63effe0f-53fb-4144-b520-9648763205ae admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.232704 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63effe0f-53fb-4144-b520-9648763205ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.235382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63effe0f-53fb-4144-b520-9648763205ae admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.245018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bf00859d-193f-4dd4-8d22-218614a15a79 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:18.246228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bf00859d-193f-4dd4-8d22-218614a15a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:18.254565 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 776/1557] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:18.255330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00859d-193f-4dd4-8d22-218614a15a79 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.259194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 782/1558] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.259825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63effe0f-53fb-4144-b520-9648763205ae admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.264679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00859d-193f-4dd4-8d22-218614a15a79 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.269445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134547c2-ceb8-4967-a808-07b54b1cd6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.270104 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134547c2-ceb8-4967-a808-07b54b1cd6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.270228 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63effe0f-53fb-4144-b520-9648763205ae admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.270382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134547c2-ceb8-4967-a808-07b54b1cd6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.270646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134547c2-ceb8-4967-a808-07b54b1cd6c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.271395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134547c2-ceb8-4967-a808-07b54b1cd6c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:18.271667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134547c2-ceb8-4967-a808-07b54b1cd6c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.273298 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134547c2-ceb8-4967-a808-07b54b1cd6c3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.274637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534c978-7e6b-4b41-bf44-480830e172d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.275280 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534c978-7e6b-4b41-bf44-480830e172d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.275950 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534c978-7e6b-4b41-bf44-480830e172d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.275950 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534c978-7e6b-4b41-bf44-480830e172d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.276570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534c978-7e6b-4b41-bf44-480830e172d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:18.276900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534c978-7e6b-4b41-bf44-480830e172d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.278472 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534c978-7e6b-4b41-bf44-480830e172d8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.285136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 777/1559] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:18.286264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134547c2-ceb8-4967-a808-07b54b1cd6c3 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.296566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 783/1560] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:18.296566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534c978-7e6b-4b41-bf44-480830e172d8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.302219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534c978-7e6b-4b41-bf44-480830e172d8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.303278 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134547c2-ceb8-4967-a808-07b54b1cd6c3 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.306610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4ae92-d7ea-44c3-8403-ada390ea66e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.307274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4ae92-d7ea-44c3-8403-ada390ea66e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.307559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4ae92-d7ea-44c3-8403-ada390ea66e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.307806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4ae92-d7ea-44c3-8403-ada390ea66e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.308013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f5fed-bce8-42ed-8a27-7dfcc1d3f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.308503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4ae92-d7ea-44c3-8403-ada390ea66e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.308705 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f5fed-bce8-42ed-8a27-7dfcc1d3f0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.308995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f5fed-bce8-42ed-8a27-7dfcc1d3f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.309270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f5fed-bce8-42ed-8a27-7dfcc1d3f0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.309507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-b4d4ae92-d7ea-44c3-8403-ada390ea66e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:43:18.309507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Apr 29 18:43:18.309507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: Apr 29 18:43:18.309956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f5fed-bce8-42ed-8a27-7dfcc1d3f0f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.310654 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4ae92-d7ea-44c3-8403-ada390ea66e0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.311974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f5fed-bce8-42ed-8a27-7dfcc1d3f0f8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.337551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 784/1561] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.342046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 778/1562] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.342192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2986b-45a3-4015-9511-c29421a0bdbe None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.342296 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2986b-45a3-4015-9511-c29421a0bdbe None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.342506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2986b-45a3-4015-9511-c29421a0bdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.344602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f5fed-bce8-42ed-8a27-7dfcc1d3f0f8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.360287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ddb2986b-45a3-4015-9511-c29421a0bdbe None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:18.361592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ddb2986b-45a3-4015-9511-c29421a0bdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:18.403241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f5fed-bce8-42ed-8a27-7dfcc1d3f0f8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.407883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d895a0e-91f2-4127-a4d7-46e8095c7567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.408639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d895a0e-91f2-4127-a4d7-46e8095c7567 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.408919 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d895a0e-91f2-4127-a4d7-46e8095c7567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.409189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d895a0e-91f2-4127-a4d7-46e8095c7567 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.409911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d895a0e-91f2-4127-a4d7-46e8095c7567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:18.410209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d895a0e-91f2-4127-a4d7-46e8095c7567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.411844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d895a0e-91f2-4127-a4d7-46e8095c7567 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.420414 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 785/1563] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:18.421507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2986b-45a3-4015-9511-c29421a0bdbe None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.430060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 779/1564] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:18.430060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e4e7-cfa1-44fc-b8aa-e42648ee65e4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.430060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e4e7-cfa1-44fc-b8aa-e42648ee65e4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.430060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e4e7-cfa1-44fc-b8aa-e42648ee65e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.431068 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2986b-45a3-4015-9511-c29421a0bdbe None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.435638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e8bb1-294b-4742-9c41-896b5d7cf632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.436295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e8bb1-294b-4742-9c41-896b5d7cf632 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.436687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e8bb1-294b-4742-9c41-896b5d7cf632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.436963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e8bb1-294b-4742-9c41-896b5d7cf632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.437685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e8bb1-294b-4742-9c41-896b5d7cf632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.439295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e8bb1-294b-4742-9c41-896b5d7cf632 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.443164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8012e4e7-cfa1-44fc-b8aa-e42648ee65e4 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:18.444349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8012e4e7-cfa1-44fc-b8aa-e42648ee65e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:18.449089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 786/1565] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:18.450701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e8bb1-294b-4742-9c41-896b5d7cf632 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.452406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 780/1566] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:18.453632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012e4e7-cfa1-44fc-b8aa-e42648ee65e4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.461339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e8bb1-294b-4742-9c41-896b5d7cf632 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.462394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012e4e7-cfa1-44fc-b8aa-e42648ee65e4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.466024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e4cc1-e1cd-4e6a-b68d-60dc57d6c1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.466744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e4cc1-e1cd-4e6a-b68d-60dc57d6c1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.466855 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b46e3-3af5-4c69-a346-971082985a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.467149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e4cc1-e1cd-4e6a-b68d-60dc57d6c1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.467464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b46e3-3af5-4c69-a346-971082985a32 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.467597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e4cc1-e1cd-4e6a-b68d-60dc57d6c1d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.467814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b46e3-3af5-4c69-a346-971082985a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.468043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b46e3-3af5-4c69-a346-971082985a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.468253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e4cc1-e1cd-4e6a-b68d-60dc57d6c1d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.468735 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b46e3-3af5-4c69-a346-971082985a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.470394 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e4cc1-e1cd-4e6a-b68d-60dc57d6c1d9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.470636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b46e3-3af5-4c69-a346-971082985a32 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.498143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 781/1567] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/projects => generated 403 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.498143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b46e3-3af5-4c69-a346-971082985a32 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.498453 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 787/1568] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.499894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e4cc1-e1cd-4e6a-b68d-60dc57d6c1d9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.554800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b46e3-3af5-4c69-a346-971082985a32 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.557004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e4cc1-e1cd-4e6a-b68d-60dc57d6c1d9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.559663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58811f7b-a23b-420b-90a3-00a491d1653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.560335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58811f7b-a23b-420b-90a3-00a491d1653a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.560654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58811f7b-a23b-420b-90a3-00a491d1653a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.560935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58811f7b-a23b-420b-90a3-00a491d1653a admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.561910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a06ed4-bc0c-4d82-9a04-4e133c4c7d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.562075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58811f7b-a23b-420b-90a3-00a491d1653a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.562655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06ed4-bc0c-4d82-9a04-4e133c4c7d7b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.562925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-58811f7b-a23b-420b-90a3-00a491d1653a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:43:18.562925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Apr 29 18:43:18.562925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: Apr 29 18:43:18.563267 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06ed4-bc0c-4d82-9a04-4e133c4c7d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.563267 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06ed4-bc0c-4d82-9a04-4e133c4c7d7b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.563860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a06ed4-bc0c-4d82-9a04-4e133c4c7d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.564121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58811f7b-a23b-420b-90a3-00a491d1653a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.565789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a06ed4-bc0c-4d82-9a04-4e133c4c7d7b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.584831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 782/1569] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.585640 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58811f7b-a23b-420b-90a3-00a491d1653a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.589100 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 788/1570] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/users => generated 295 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.589100 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a06ed4-bc0c-4d82-9a04-4e133c4c7d7b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.596278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a06ed4-bc0c-4d82-9a04-4e133c4c7d7b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.596863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58811f7b-a23b-420b-90a3-00a491d1653a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.601669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65502d16-fe95-491a-a90b-ba264d0da087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.602335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65502d16-fe95-491a-a90b-ba264d0da087 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.602638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65502d16-fe95-491a-a90b-ba264d0da087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.602909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65502d16-fe95-491a-a90b-ba264d0da087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.603602 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65502d16-fe95-491a-a90b-ba264d0da087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.605885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65502d16-fe95-491a-a90b-ba264d0da087 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.607049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192237c-1749-43d0-ab0a-5ac634169757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.607792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192237c-1749-43d0-ab0a-5ac634169757 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.608045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192237c-1749-43d0-ab0a-5ac634169757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.608322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192237c-1749-43d0-ab0a-5ac634169757 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.609367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192237c-1749-43d0-ab0a-5ac634169757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:18.609808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192237c-1749-43d0-ab0a-5ac634169757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.611419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192237c-1749-43d0-ab0a-5ac634169757 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.623570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 783/1571] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:18.624587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192237c-1749-43d0-ab0a-5ac634169757 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.635435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192237c-1749-43d0-ab0a-5ac634169757 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.639949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9453-589c-41a7-8f0c-c3feb2b85c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.640711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9453-589c-41a7-8f0c-c3feb2b85c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.640858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 789/1572] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.640988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9453-589c-41a7-8f0c-c3feb2b85c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.641261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9453-589c-41a7-8f0c-c3feb2b85c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.641779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65502d16-fe95-491a-a90b-ba264d0da087 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.641973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc9453-589c-41a7-8f0c-c3feb2b85c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.643618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc9453-589c-41a7-8f0c-c3feb2b85c69 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.654656 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 784/1573] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:18.655827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9453-589c-41a7-8f0c-c3feb2b85c69 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.664911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9453-589c-41a7-8f0c-c3feb2b85c69 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.706642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65502d16-fe95-491a-a90b-ba264d0da087 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.711784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc14917-5d95-4157-bd6e-089dbc3aa6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.712448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc14917-5d95-4157-bd6e-089dbc3aa6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.712767 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc14917-5d95-4157-bd6e-089dbc3aa6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.713038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc14917-5d95-4157-bd6e-089dbc3aa6b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.713765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc14917-5d95-4157-bd6e-089dbc3aa6b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:18.714070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc14917-5d95-4157-bd6e-089dbc3aa6b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.715653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc14917-5d95-4157-bd6e-089dbc3aa6b1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.718975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d9e2f-52d5-42cf-832f-0e1aeeb4c629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.719645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d9e2f-52d5-42cf-832f-0e1aeeb4c629 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.719923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d9e2f-52d5-42cf-832f-0e1aeeb4c629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.720189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d9e2f-52d5-42cf-832f-0e1aeeb4c629 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.720901 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d9e2f-52d5-42cf-832f-0e1aeeb4c629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.722800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d9e2f-52d5-42cf-832f-0e1aeeb4c629 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.727108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 790/1574] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:18.731400 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc14917-5d95-4157-bd6e-089dbc3aa6b1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.743343 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc14917-5d95-4157-bd6e-089dbc3aa6b1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.745428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 785/1575] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.746251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d9e2f-52d5-42cf-832f-0e1aeeb4c629 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.747909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127508b-fe73-4414-8e98-5d6f0ad0b49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.748596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127508b-fe73-4414-8e98-5d6f0ad0b49b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.748877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127508b-fe73-4414-8e98-5d6f0ad0b49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.749248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127508b-fe73-4414-8e98-5d6f0ad0b49b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.749891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127508b-fe73-4414-8e98-5d6f0ad0b49b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.751998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127508b-fe73-4414-8e98-5d6f0ad0b49b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.758194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d9e2f-52d5-42cf-832f-0e1aeeb4c629 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.762901 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d7950-a2e3-455e-ab95-73bd74d77f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.763563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d7950-a2e3-455e-ab95-73bd74d77f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.763839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d7950-a2e3-455e-ab95-73bd74d77f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.764108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d7950-a2e3-455e-ab95-73bd74d77f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.764904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d7950-a2e3-455e-ab95-73bd74d77f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.766476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d7950-a2e3-455e-ab95-73bd74d77f2c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.772966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 791/1576] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.773797 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127508b-fe73-4414-8e98-5d6f0ad0b49b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.779526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 786/1577] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:18.780384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d7950-a2e3-455e-ab95-73bd74d77f2c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.783685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127508b-fe73-4414-8e98-5d6f0ad0b49b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.789148 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ef603-715c-4113-bbcf-fead09b33074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.789561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d7950-a2e3-455e-ab95-73bd74d77f2c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.789815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ef603-715c-4113-bbcf-fead09b33074 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.790099 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ef603-715c-4113-bbcf-fead09b33074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.790370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ef603-715c-4113-bbcf-fead09b33074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.791091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ef603-715c-4113-bbcf-fead09b33074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.792755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ef603-715c-4113-bbcf-fead09b33074 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.794297 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5de6-957b-473b-b0f6-b8b57a298ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.794926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5de6-957b-473b-b0f6-b8b57a298ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.795203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5de6-957b-473b-b0f6-b8b57a298ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.795471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5de6-957b-473b-b0f6-b8b57a298ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.796189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d5de6-957b-473b-b0f6-b8b57a298ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.798064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d5de6-957b-473b-b0f6-b8b57a298ec7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.806742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 792/1578] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:18.806742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ef603-715c-4113-bbcf-fead09b33074 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.816559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ef603-715c-4113-bbcf-fead09b33074 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.821276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd7283-c466-4b25-b1e3-4635fc08cf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.821932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd7283-c466-4b25-b1e3-4635fc08cf56 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.822210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd7283-c466-4b25-b1e3-4635fc08cf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.822481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd7283-c466-4b25-b1e3-4635fc08cf56 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.823157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd7283-c466-4b25-b1e3-4635fc08cf56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.826582 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 787/1579] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.826582 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5de6-957b-473b-b0f6-b8b57a298ec7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.826808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd7283-c466-4b25-b1e3-4635fc08cf56 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.848745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 793/1580] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.850103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd7283-c466-4b25-b1e3-4635fc08cf56 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.861064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd7283-c466-4b25-b1e3-4635fc08cf56 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.883550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5de6-957b-473b-b0f6-b8b57a298ec7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:18.888063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014a952-9b58-4fba-abf7-cdad0b58ddda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.889001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014a952-9b58-4fba-abf7-cdad0b58ddda admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.889282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014a952-9b58-4fba-abf7-cdad0b58ddda admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.889571 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014a952-9b58-4fba-abf7-cdad0b58ddda admin admin] PATH_INFO: `/v3/projects/609e205e19d14496af1b90aa48697c44/users/bb050519bc3547378364d107f054bebf/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.903593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014a952-9b58-4fba-abf7-cdad0b58ddda admin admin] RBAC: Authorizing `identity:create_grant(project_id=609e205e19d14496af1b90aa48697c44, user_id=bb050519bc3547378364d107f054bebf, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.906663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014a952-9b58-4fba-abf7-cdad0b58ddda admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.915638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67389fa2-6bde-4481-b661-1fe138ca149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:18.916288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67389fa2-6bde-4481-b661-1fe138ca149a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:18.916599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67389fa2-6bde-4481-b661-1fe138ca149a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:18.916869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67389fa2-6bde-4481-b661-1fe138ca149a admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:18.917559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67389fa2-6bde-4481-b661-1fe138ca149a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:18.919427 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67389fa2-6bde-4481-b661-1fe138ca149a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:18.920777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 788/1581] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:18 2022] PUT /identity/v3/projects/609e205e19d14496af1b90aa48697c44/users/bb050519bc3547378364d107f054bebf/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:18.921654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014a952-9b58-4fba-abf7-cdad0b58ddda admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.948162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 794/1582] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:18 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:18.948162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67389fa2-6bde-4481-b661-1fe138ca149a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:18.981325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014a952-9b58-4fba-abf7-cdad0b58ddda admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.002768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67389fa2-6bde-4481-b661-1fe138ca149a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.007442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90614f7-10d9-4a2a-8ff9-47d087833f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.008183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90614f7-10d9-4a2a-8ff9-47d087833f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.008557 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90614f7-10d9-4a2a-8ff9-47d087833f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.008837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90614f7-10d9-4a2a-8ff9-47d087833f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.009589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90614f7-10d9-4a2a-8ff9-47d087833f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.011285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90614f7-10d9-4a2a-8ff9-47d087833f3d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.021372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 795/1583] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.022161 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90614f7-10d9-4a2a-8ff9-47d087833f3d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.031678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90614f7-10d9-4a2a-8ff9-47d087833f3d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.036221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a73e0c-02f1-4998-adbd-f00295c3fdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.036537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef01c32-2866-4bdb-bcc7-e29e62f6fafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.036891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a73e0c-02f1-4998-adbd-f00295c3fdd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.037174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a73e0c-02f1-4998-adbd-f00295c3fdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.037330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef01c32-2866-4bdb-bcc7-e29e62f6fafc admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.037488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a73e0c-02f1-4998-adbd-f00295c3fdd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.037614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef01c32-2866-4bdb-bcc7-e29e62f6fafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.037878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef01c32-2866-4bdb-bcc7-e29e62f6fafc admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.038155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a73e0c-02f1-4998-adbd-f00295c3fdd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.038684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef01c32-2866-4bdb-bcc7-e29e62f6fafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.039719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a73e0c-02f1-4998-adbd-f00295c3fdd8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.040293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef01c32-2866-4bdb-bcc7-e29e62f6fafc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.049471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 796/1584] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.050322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a73e0c-02f1-4998-adbd-f00295c3fdd8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.050773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 789/1585] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.051605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef01c32-2866-4bdb-bcc7-e29e62f6fafc admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.059850 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a73e0c-02f1-4998-adbd-f00295c3fdd8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.060919 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef01c32-2866-4bdb-bcc7-e29e62f6fafc admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.064530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c041647-b6d7-4a51-b5fe-fca99ab3882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.065169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c041647-b6d7-4a51-b5fe-fca99ab3882c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.065448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c041647-b6d7-4a51-b5fe-fca99ab3882c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.065720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c041647-b6d7-4a51-b5fe-fca99ab3882c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.065905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bb4bd-1e15-41fd-832c-2f4866b1ba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.066424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c041647-b6d7-4a51-b5fe-fca99ab3882c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.066552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bb4bd-1e15-41fd-832c-2f4866b1ba17 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.066832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bb4bd-1e15-41fd-832c-2f4866b1ba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.067103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bb4bd-1e15-41fd-832c-2f4866b1ba17 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.067780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bb4bd-1e15-41fd-832c-2f4866b1ba17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.068001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c041647-b6d7-4a51-b5fe-fca99ab3882c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.069699 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bb4bd-1e15-41fd-832c-2f4866b1ba17 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.078788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 797/1586] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.079667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c041647-b6d7-4a51-b5fe-fca99ab3882c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.089463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c041647-b6d7-4a51-b5fe-fca99ab3882c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.091490 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 790/1587] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:19 2022] POST /identity/v3/users => generated 296 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:19.092860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760b1f5-ce9d-4873-9579-2def734455f3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.093151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760b1f5-ce9d-4873-9579-2def734455f3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.093458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760b1f5-ce9d-4873-9579-2def734455f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.096905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62181251-5178-4127-a979-69ea380e7033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.097560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62181251-5178-4127-a979-69ea380e7033 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.097841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62181251-5178-4127-a979-69ea380e7033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.098112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62181251-5178-4127-a979-69ea380e7033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.098827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62181251-5178-4127-a979-69ea380e7033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.100411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62181251-5178-4127-a979-69ea380e7033 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.110804 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 798/1588] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.111732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62181251-5178-4127-a979-69ea380e7033 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.122944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62181251-5178-4127-a979-69ea380e7033 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.126058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-2760b1f5-ce9d-4873-9579-2def734455f3 None None] MFA Rules not processed for user `bb050519bc3547378364d107f054bebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:19.127268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2760b1f5-ce9d-4873-9579-2def734455f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:19.128124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e726be5-976b-4966-8f49-8861a543c1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.128822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e726be5-976b-4966-8f49-8861a543c1e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.129105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e726be5-976b-4966-8f49-8861a543c1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.129376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e726be5-976b-4966-8f49-8861a543c1e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.130069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e726be5-976b-4966-8f49-8861a543c1e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.132939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e726be5-976b-4966-8f49-8861a543c1e9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.157087 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 799/1589] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:19 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:19.157846 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e726be5-976b-4966-8f49-8861a543c1e9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.167826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e726be5-976b-4966-8f49-8861a543c1e9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.173048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171b931-8ccb-4099-9c09-b04bb9b805e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.173715 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171b931-8ccb-4099-9c09-b04bb9b805e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.173992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171b931-8ccb-4099-9c09-b04bb9b805e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.174262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171b931-8ccb-4099-9c09-b04bb9b805e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.174990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171b931-8ccb-4099-9c09-b04bb9b805e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.176620 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171b931-8ccb-4099-9c09-b04bb9b805e1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.188463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 800/1590] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.189041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171b931-8ccb-4099-9c09-b04bb9b805e1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.204215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 791/1591] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:19.204513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171b931-8ccb-4099-9c09-b04bb9b805e1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.204642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760b1f5-ce9d-4873-9579-2def734455f3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.204761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4b0a7-6961-4154-9d17-814d1ce11691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.205671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4b0a7-6961-4154-9d17-814d1ce11691 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.205936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4b0a7-6961-4154-9d17-814d1ce11691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.206198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4b0a7-6961-4154-9d17-814d1ce11691 admin admin] PATH_INFO: `/v3/projects/8d9c212254b240af938e5d7c2e9e1261/users/64c8eec4efab42bfa5d9e9b157883dd7/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.210358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760b1f5-ce9d-4873-9579-2def734455f3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.215225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc209ba-8575-412c-9876-a7983c3a50e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.216535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc209ba-8575-412c-9876-a7983c3a50e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.216535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc209ba-8575-412c-9876-a7983c3a50e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.216776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc209ba-8575-412c-9876-a7983c3a50e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.217320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc209ba-8575-412c-9876-a7983c3a50e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.219124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc209ba-8575-412c-9876-a7983c3a50e6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.223464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4b0a7-6961-4154-9d17-814d1ce11691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9c212254b240af938e5d7c2e9e1261, user_id=64c8eec4efab42bfa5d9e9b157883dd7, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.224585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-33b4b0a7-6961-4154-9d17-814d1ce11691 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:43:19.224585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Apr 29 18:43:19.224585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: Apr 29 18:43:19.226495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4b0a7-6961-4154-9d17-814d1ce11691 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.229881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 792/1592] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.230649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc209ba-8575-412c-9876-a7983c3a50e6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.241447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc209ba-8575-412c-9876-a7983c3a50e6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.243806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 801/1593] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:19 2022] PUT /identity/v3/projects/8d9c212254b240af938e5d7c2e9e1261/users/64c8eec4efab42bfa5d9e9b157883dd7/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:19.244954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4b0a7-6961-4154-9d17-814d1ce11691 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.302439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea38f6-7104-4c9a-ab48-fa6e1b51029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.303496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea38f6-7104-4c9a-ab48-fa6e1b51029b admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.303775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea38f6-7104-4c9a-ab48-fa6e1b51029b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.304045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea38f6-7104-4c9a-ab48-fa6e1b51029b admin admin] PATH_INFO: `/v3/projects/e9f31cfe6a484184b4b4873bb918b266/users/49aaea67fba64d2583d2023505ce7779/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.307760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4b0a7-6961-4154-9d17-814d1ce11691 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.312531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a85b6-df90-49cb-a1a1-ec4994894465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.313186 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a85b6-df90-49cb-a1a1-ec4994894465 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.313683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a85b6-df90-49cb-a1a1-ec4994894465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.313963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a85b6-df90-49cb-a1a1-ec4994894465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.314696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a85b6-df90-49cb-a1a1-ec4994894465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.316385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a85b6-df90-49cb-a1a1-ec4994894465 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.318539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea38f6-7104-4c9a-ab48-fa6e1b51029b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f31cfe6a484184b4b4873bb918b266, user_id=49aaea67fba64d2583d2023505ce7779, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.321198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea38f6-7104-4c9a-ab48-fa6e1b51029b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.326976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 802/1594] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.328906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a85b6-df90-49cb-a1a1-ec4994894465 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.338214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 793/1595] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:19 2022] PUT /identity/v3/projects/e9f31cfe6a484184b4b4873bb918b266/users/49aaea67fba64d2583d2023505ce7779/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:19.339334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a85b6-df90-49cb-a1a1-ec4994894465 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.342479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea38f6-7104-4c9a-ab48-fa6e1b51029b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.396733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967a903-be3f-47f6-b2e5-c21806fac818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.397439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967a903-be3f-47f6-b2e5-c21806fac818 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.397725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967a903-be3f-47f6-b2e5-c21806fac818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.398003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967a903-be3f-47f6-b2e5-c21806fac818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.398728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967a903-be3f-47f6-b2e5-c21806fac818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.400370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967a903-be3f-47f6-b2e5-c21806fac818 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.403156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea38f6-7104-4c9a-ab48-fa6e1b51029b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.408064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576c572-baba-4fbb-8233-874622e95aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.408796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576c572-baba-4fbb-8233-874622e95aac admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.409082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576c572-baba-4fbb-8233-874622e95aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.409355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576c572-baba-4fbb-8233-874622e95aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.410107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576c572-baba-4fbb-8233-874622e95aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.410274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 803/1596] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.411042 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967a903-be3f-47f6-b2e5-c21806fac818 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.411724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576c572-baba-4fbb-8233-874622e95aac admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.420769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967a903-be3f-47f6-b2e5-c21806fac818 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.424451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 794/1597] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.424451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b946181-d99e-4f8c-8199-c7e2568a8f13 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.424451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b946181-d99e-4f8c-8199-c7e2568a8f13 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.424451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b946181-d99e-4f8c-8199-c7e2568a8f13 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.425166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 795/1598] 10.222.0.50 () {54 vars in 960 bytes} [Fri Apr 29 18:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:43:19.425913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b946181-d99e-4f8c-8199-c7e2568a8f13 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.428972 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184226e-bc5d-4733-884f-e72a7473ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.429867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184226e-bc5d-4733-884f-e72a7473ce77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.430150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184226e-bc5d-4733-884f-e72a7473ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.430429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184226e-bc5d-4733-884f-e72a7473ce77 admin admin] PATH_INFO: `/v3/projects/3b376e970ee94348aa59b7e1792baa7e/users/4570168001784e9eab4711bf23afb47b/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.438891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b946181-d99e-4f8c-8199-c7e2568a8f13 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.443393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce245262-3f0a-4fed-ac58-5b565b796da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.444251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245262-3f0a-4fed-ac58-5b565b796da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.444595 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245262-3f0a-4fed-ac58-5b565b796da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.444832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245262-3f0a-4fed-ac58-5b565b796da4 admin admin] PATH_INFO: `/v3/projects/5c9c70cdc1a247a2b79561a4b202a277/users/94716f9f6776435ba260230e02c39718/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.445369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184226e-bc5d-4733-884f-e72a7473ce77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b376e970ee94348aa59b7e1792baa7e, user_id=4570168001784e9eab4711bf23afb47b, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.448091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184226e-bc5d-4733-884f-e72a7473ce77 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.460407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce245262-3f0a-4fed-ac58-5b565b796da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c9c70cdc1a247a2b79561a4b202a277, user_id=94716f9f6776435ba260230e02c39718, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.461816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 804/1599] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:19 2022] PUT /identity/v3/projects/3b376e970ee94348aa59b7e1792baa7e/users/4570168001784e9eab4711bf23afb47b/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:19.463130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e4157-95d4-43c9-a36e-3177e872df80 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.463241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce245262-3f0a-4fed-ac58-5b565b796da4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.463429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e4157-95d4-43c9-a36e-3177e872df80 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.463681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e4157-95d4-43c9-a36e-3177e872df80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.476145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 796/1600] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:19 2022] PUT /identity/v3/projects/5c9c70cdc1a247a2b79561a4b202a277/users/94716f9f6776435ba260230e02c39718/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:19.480602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce245262-3f0a-4fed-ac58-5b565b796da4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.480989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b52e4157-95d4-43c9-a36e-3177e872df80 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:19.482234 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b52e4157-95d4-43c9-a36e-3177e872df80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:19.544146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce245262-3f0a-4fed-ac58-5b565b796da4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.549342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94b342-4334-44e8-865b-8addb41ec975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.550153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94b342-4334-44e8-865b-8addb41ec975 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.550706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94b342-4334-44e8-865b-8addb41ec975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.550998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94b342-4334-44e8-865b-8addb41ec975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.552327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94b342-4334-44e8-865b-8addb41ec975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.554262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 805/1601] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:19.555859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e612e07-d68d-42c1-b602-451bbee45866 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.556155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e612e07-d68d-42c1-b602-451bbee45866 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.556431 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e612e07-d68d-42c1-b602-451bbee45866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.558386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94b342-4334-44e8-865b-8addb41ec975 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.569756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 797/1602] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.570457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94b342-4334-44e8-865b-8addb41ec975 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.581014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8e612e07-d68d-42c1-b602-451bbee45866 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:19.581188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94b342-4334-44e8-865b-8addb41ec975 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.582258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8e612e07-d68d-42c1-b602-451bbee45866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:19.586598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb96cd-3158-446b-a2ce-ab0f0b6760f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.587487 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb96cd-3158-446b-a2ce-ab0f0b6760f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.587762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb96cd-3158-446b-a2ce-ab0f0b6760f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.588031 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb96cd-3158-446b-a2ce-ab0f0b6760f0 admin admin] PATH_INFO: `/v3/projects/1775b09c31a0480a99ba335c2fddd9bc/users/ffc1b3f143494a4a91e81729b7bd0754/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.591357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 806/1603] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:19.592148 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e612e07-d68d-42c1-b602-451bbee45866 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.604656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e612e07-d68d-42c1-b602-451bbee45866 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.606669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb96cd-3158-446b-a2ce-ab0f0b6760f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1775b09c31a0480a99ba335c2fddd9bc, user_id=ffc1b3f143494a4a91e81729b7bd0754, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.609649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce983d3-d597-4575-ad4d-83858b963ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.610321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce983d3-d597-4575-ad4d-83858b963ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.610454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb96cd-3158-446b-a2ce-ab0f0b6760f0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.610591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce983d3-d597-4575-ad4d-83858b963ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.610860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce983d3-d597-4575-ad4d-83858b963ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.611604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce983d3-d597-4575-ad4d-83858b963ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.613275 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce983d3-d597-4575-ad4d-83858b963ffb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.627444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 798/1604] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:19 2022] PUT /identity/v3/projects/1775b09c31a0480a99ba335c2fddd9bc/users/ffc1b3f143494a4a91e81729b7bd0754/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:19.628765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2571a-9ed4-4b65-842f-c3fe84735c29 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.629053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2571a-9ed4-4b65-842f-c3fe84735c29 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.629319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2571a-9ed4-4b65-842f-c3fe84735c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.633623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 807/1605] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:19.634776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863604e4-e0b6-4ca7-8771-e9ded1a108ab None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.635061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863604e4-e0b6-4ca7-8771-e9ded1a108ab None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.635332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863604e4-e0b6-4ca7-8771-e9ded1a108ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.652252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f5d2571a-9ed4-4b65-842f-c3fe84735c29 None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:19.652595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-863604e4-e0b6-4ca7-8771-e9ded1a108ab None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:19.653511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f5d2571a-9ed4-4b65-842f-c3fe84735c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:19.653760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-863604e4-e0b6-4ca7-8771-e9ded1a108ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:19.712550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 808/1606] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:19.713153 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5999a-ee91-4428-9052-1cd62c15428b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.713932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5999a-ee91-4428-9052-1cd62c15428b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.714467 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5999a-ee91-4428-9052-1cd62c15428b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.723297 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 799/1607] 10.222.0.50 () {58 vars in 1057 bytes} [Fri Apr 29 18:43:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:19.729636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2571a-9ed4-4b65-842f-c3fe84735c29 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.732596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b9a5999a-ee91-4428-9052-1cd62c15428b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:19.733847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b9a5999a-ee91-4428-9052-1cd62c15428b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:19.744425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 809/1608] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:19.744425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5999a-ee91-4428-9052-1cd62c15428b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.753749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5999a-ee91-4428-9052-1cd62c15428b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.758339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c0e47-568e-4d1a-8772-63dd3c645032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.759354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c0e47-568e-4d1a-8772-63dd3c645032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.759627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c0e47-568e-4d1a-8772-63dd3c645032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.759891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c0e47-568e-4d1a-8772-63dd3c645032 admin admin] PATH_INFO: `/v3/projects/e2f2dec75acb4277945a986b8393b0a1/users/e9a2443e71c54075823f75fdb8df469e/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.774101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c0e47-568e-4d1a-8772-63dd3c645032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f2dec75acb4277945a986b8393b0a1, user_id=e9a2443e71c54075823f75fdb8df469e, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.776749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c0e47-568e-4d1a-8772-63dd3c645032 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.784295 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2571a-9ed4-4b65-842f-c3fe84735c29 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.787935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512ae4-0320-4587-92d6-7daa7d0e812a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.788612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512ae4-0320-4587-92d6-7daa7d0e812a service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.788889 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512ae4-0320-4587-92d6-7daa7d0e812a service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.789156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e512ae4-0320-4587-92d6-7daa7d0e812a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.791955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 810/1609] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:19 2022] PUT /identity/v3/projects/e2f2dec75acb4277945a986b8393b0a1/users/e9a2443e71c54075823f75fdb8df469e/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:19.792591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c0e47-568e-4d1a-8772-63dd3c645032 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.798776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512ae4-0320-4587-92d6-7daa7d0e812a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.800220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e512ae4-0320-4587-92d6-7daa7d0e812a service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.809600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 800/1610] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:19.810426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512ae4-0320-4587-92d6-7daa7d0e812a service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.819752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e512ae4-0320-4587-92d6-7daa7d0e812a service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.847664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c0e47-568e-4d1a-8772-63dd3c645032 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.851309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43997d2-c0f5-4685-944b-dad8fc729f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.852032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43997d2-c0f5-4685-944b-dad8fc729f92 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.852307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43997d2-c0f5-4685-944b-dad8fc729f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.852674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43997d2-c0f5-4685-944b-dad8fc729f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.882884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa21ce-32ff-44bb-9515-d000a28f8ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.883795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa21ce-32ff-44bb-9515-d000a28f8ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.884110 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa21ce-32ff-44bb-9515-d000a28f8ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.884409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa21ce-32ff-44bb-9515-d000a28f8ff0 admin admin] PATH_INFO: `/v3/projects/93c51c2ae9f342c3be4a8029be23f7a3/users/965be658862043b98d2411b85b35f81b/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.902310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa21ce-32ff-44bb-9515-d000a28f8ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c51c2ae9f342c3be4a8029be23f7a3, user_id=965be658862043b98d2411b85b35f81b, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.906714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa21ce-32ff-44bb-9515-d000a28f8ff0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.922962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43997d2-c0f5-4685-944b-dad8fc729f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:19.932108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 801/1611] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:19 2022] PUT /identity/v3/projects/93c51c2ae9f342c3be4a8029be23f7a3/users/965be658862043b98d2411b85b35f81b/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:19.932417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43997d2-c0f5-4685-944b-dad8fc729f92 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:19.932549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48399e-53e0-455f-bdcd-04cbcbb2e7b6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.932549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48399e-53e0-455f-bdcd-04cbcbb2e7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.932549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48399e-53e0-455f-bdcd-04cbcbb2e7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:19.935303 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 811/1612] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:19.936212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43997d2-c0f5-4685-944b-dad8fc729f92 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:19.944530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43997d2-c0f5-4685-944b-dad8fc729f92 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:19.963622 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7a48399e-53e0-455f-bdcd-04cbcbb2e7b6 None None] MFA Rules not processed for user `ffc1b3f143494a4a91e81729b7bd0754`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:19.964936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7a48399e-53e0-455f-bdcd-04cbcbb2e7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:19.994985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d936fc-b37a-4dfe-b46f-2b666c2cd15a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:19.995714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d936fc-b37a-4dfe-b46f-2b666c2cd15a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:19.996127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d936fc-b37a-4dfe-b46f-2b666c2cd15a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:19.996385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d936fc-b37a-4dfe-b46f-2b666c2cd15a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.036244 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 802/1613] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:20.037121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48399e-53e0-455f-bdcd-04cbcbb2e7b6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:20.066181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d936fc-b37a-4dfe-b46f-2b666c2cd15a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:20.068013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d936fc-b37a-4dfe-b46f-2b666c2cd15a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:20.078746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 812/1614] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:20.079424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d936fc-b37a-4dfe-b46f-2b666c2cd15a service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:20.087755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d936fc-b37a-4dfe-b46f-2b666c2cd15a service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:20.092394 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c52822-f1b9-4dde-8949-e9bd86534f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:20.093137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c52822-f1b9-4dde-8949-e9bd86534f3a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.093425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c52822-f1b9-4dde-8949-e9bd86534f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.093725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c52822-f1b9-4dde-8949-e9bd86534f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.105439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c52822-f1b9-4dde-8949-e9bd86534f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:20.107316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c52822-f1b9-4dde-8949-e9bd86534f3a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:20.118162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 813/1615] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:20.119481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf7519-abc5-4682-88b0-fe635e163b01 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.119773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf7519-abc5-4682-88b0-fe635e163b01 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.120048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf7519-abc5-4682-88b0-fe635e163b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.136829 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48399e-53e0-455f-bdcd-04cbcbb2e7b6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:20.140717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-32bf7519-abc5-4682-88b0-fe635e163b01 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:20.141844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-32bf7519-abc5-4682-88b0-fe635e163b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:20.145418 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c02fe-4f3e-4948-a04b-ee8dcc36538d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:20.146327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c02fe-4f3e-4948-a04b-ee8dcc36538d service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.146699 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c02fe-4f3e-4948-a04b-ee8dcc36538d service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.147181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c02fe-4f3e-4948-a04b-ee8dcc36538d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.151273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 814/1616] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:20.152874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a955a-466a-4785-ae6e-2aba846db0db None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.153332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a955a-466a-4785-ae6e-2aba846db0db None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.153768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a955a-466a-4785-ae6e-2aba846db0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.214309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ef6a955a-466a-4785-ae6e-2aba846db0db None None] MFA Rules not processed for user `965be658862043b98d2411b85b35f81b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:20.215561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ef6a955a-466a-4785-ae6e-2aba846db0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:20.303110 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c02fe-4f3e-4948-a04b-ee8dcc36538d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:20.305796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c02fe-4f3e-4948-a04b-ee8dcc36538d service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:20.338550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 803/1617] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:20.340023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d420ced-0e8a-4fc7-af9a-1b3579328a53 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.340333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d420ced-0e8a-4fc7-af9a-1b3579328a53 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.343452 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d420ced-0e8a-4fc7-af9a-1b3579328a53 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.347096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 804/1618] 10.222.0.50 () {54 vars in 960 bytes} [Fri Apr 29 18:43:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:43:20.347915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d420ced-0e8a-4fc7-af9a-1b3579328a53 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:20.373715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d420ced-0e8a-4fc7-af9a-1b3579328a53 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:20.382827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140a5c2-fb22-4324-9a89-2e50efc9d958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:20.383539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a5c2-fb22-4324-9a89-2e50efc9d958 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.385541 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a5c2-fb22-4324-9a89-2e50efc9d958 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.386899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a5c2-fb22-4324-9a89-2e50efc9d958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.406027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 815/1619] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:20.407514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dfeb0-bb5d-4180-b204-4443142017cf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.407814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dfeb0-bb5d-4180-b204-4443142017cf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.408129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dfeb0-bb5d-4180-b204-4443142017cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.412248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140a5c2-fb22-4324-9a89-2e50efc9d958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:20.413855 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140a5c2-fb22-4324-9a89-2e50efc9d958 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:20.439035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 805/1620] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:20.439825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140a5c2-fb22-4324-9a89-2e50efc9d958 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:20.448694 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2c1dfeb0-bb5d-4180-b204-4443142017cf None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:20.451294 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2c1dfeb0-bb5d-4180-b204-4443142017cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:20.459781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140a5c2-fb22-4324-9a89-2e50efc9d958 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:20.473871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56110567-484a-4279-9e07-aa4412ea223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:20.474621 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56110567-484a-4279-9e07-aa4412ea223d admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.474923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56110567-484a-4279-9e07-aa4412ea223d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.475314 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56110567-484a-4279-9e07-aa4412ea223d admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.475893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 816/1621] 10.222.0.50 () {58 vars in 1057 bytes} [Fri Apr 29 18:43:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:20.476256 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56110567-484a-4279-9e07-aa4412ea223d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:20.478340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56110567-484a-4279-9e07-aa4412ea223d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:20.481986 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dfeb0-bb5d-4180-b204-4443142017cf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:20.502684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dfeb0-bb5d-4180-b204-4443142017cf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:20.518344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e986aa-c512-4696-8931-afb9f77941dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:20.519191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e986aa-c512-4696-8931-afb9f77941dd service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.519517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e986aa-c512-4696-8931-afb9f77941dd service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.519791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e986aa-c512-4696-8931-afb9f77941dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.539748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 806/1622] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:20 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:20.541698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56110567-484a-4279-9e07-aa4412ea223d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:20.545184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e986aa-c512-4696-8931-afb9f77941dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:20.547576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e986aa-c512-4696-8931-afb9f77941dd service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:20.574361 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 817/1623] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:43:20.641853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56110567-484a-4279-9e07-aa4412ea223d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:20.648591 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4e594-f1d5-482e-8f44-6cf595278a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:20.649294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4e594-f1d5-482e-8f44-6cf595278a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.649593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4e594-f1d5-482e-8f44-6cf595278a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.649858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4e594-f1d5-482e-8f44-6cf595278a0d admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.650805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4e594-f1d5-482e-8f44-6cf595278a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:20.653504 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4e594-f1d5-482e-8f44-6cf595278a0d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:20.684171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 807/1624] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:20 2022] POST /identity/v3/users => generated 297 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:20.689338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e986aa-c512-4696-8931-afb9f77941dd service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:20.715453 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e986aa-c512-4696-8931-afb9f77941dd service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:20.728040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb1f23-b0dd-4e07-b0eb-8ccdf6c58121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:20.729216 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb1f23-b0dd-4e07-b0eb-8ccdf6c58121 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.729789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb1f23-b0dd-4e07-b0eb-8ccdf6c58121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.731044 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb1f23-b0dd-4e07-b0eb-8ccdf6c58121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.732799 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb1f23-b0dd-4e07-b0eb-8ccdf6c58121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:20.735987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb1f23-b0dd-4e07-b0eb-8ccdf6c58121 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:20.762689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 818/1625] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:20.764051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4e594-f1d5-482e-8f44-6cf595278a0d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:20.777593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5e64d-58fb-452a-8db0-b30b700d0201 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.778568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5e64d-58fb-452a-8db0-b30b700d0201 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.778939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5e64d-58fb-452a-8db0-b30b700d0201 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.781802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4e594-f1d5-482e-8f44-6cf595278a0d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:20.783397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 819/1626] 10.222.0.50 () {54 vars in 912 bytes} [Fri Apr 29 18:43:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:43:20.790155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43c01f-4958-4d03-b184-a743bb90db7b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.790700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43c01f-4958-4d03-b184-a743bb90db7b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.791287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43c01f-4958-4d03-b184-a743bb90db7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.794220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0390e1-fad0-4454-a2db-5aae167f67cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:20.796303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0390e1-fad0-4454-a2db-5aae167f67cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.796895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0390e1-fad0-4454-a2db-5aae167f67cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.798373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0390e1-fad0-4454-a2db-5aae167f67cd admin admin] PATH_INFO: `/v3/projects/9a2585b72cb444e88a1a3396ce24d559/users/887ebf2b02984ae483f26c947f6fead1/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.835064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6b43c01f-4958-4d03-b184-a743bb90db7b None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:20.837213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6b43c01f-4958-4d03-b184-a743bb90db7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:20.837370 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0390e1-fad0-4454-a2db-5aae167f67cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a2585b72cb444e88a1a3396ce24d559, user_id=887ebf2b02984ae483f26c947f6fead1, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:20.842533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0390e1-fad0-4454-a2db-5aae167f67cd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:20.882511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 808/1627] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:20 2022] PUT /identity/v3/projects/9a2585b72cb444e88a1a3396ce24d559/users/887ebf2b02984ae483f26c947f6fead1/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:20.888141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0390e1-fad0-4454-a2db-5aae167f67cd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:20.954156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 820/1628] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:43:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:43:20.955536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf81868-a63f-48a9-8e6a-4b45eed0af90 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.955821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf81868-a63f-48a9-8e6a-4b45eed0af90 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.956096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf81868-a63f-48a9-8e6a-4b45eed0af90 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.969854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 821/1629] 10.222.0.50 () {54 vars in 912 bytes} [Fri Apr 29 18:43:20 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:43:20.971300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7a8ac-8256-410c-bd51-894434174542 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.971588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7a8ac-8256-410c-bd51-894434174542 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.971864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7a8ac-8256-410c-bd51-894434174542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.986393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0390e1-fad0-4454-a2db-5aae167f67cd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:20.992387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f7b3-1a51-421d-b9cb-6417bb9c606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:20.993185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f7b3-1a51-421d-b9cb-6417bb9c606d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:20.993471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f7b3-1a51-421d-b9cb-6417bb9c606d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:20.993941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f7b3-1a51-421d-b9cb-6417bb9c606d admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:20.995294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087f7b3-1a51-421d-b9cb-6417bb9c606d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:20.997268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087f7b3-1a51-421d-b9cb-6417bb9c606d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:20.999224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0bf7a8ac-8256-410c-bd51-894434174542 None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:21.001417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0bf7a8ac-8256-410c-bd51-894434174542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:21.014836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 809/1630] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:21.016683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f7b3-1a51-421d-b9cb-6417bb9c606d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:21.152214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 822/1631] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:43:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:21.153303 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7a8ac-8256-410c-bd51-894434174542 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:21.157533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f7b3-1a51-421d-b9cb-6417bb9c606d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:21.164098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20e6ae-4719-4c81-9839-c26ba98b352b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:21.164807 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20e6ae-4719-4c81-9839-c26ba98b352b service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:21.165088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20e6ae-4719-4c81-9839-c26ba98b352b service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:21.165361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20e6ae-4719-4c81-9839-c26ba98b352b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:21.167881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7a8ac-8256-410c-bd51-894434174542 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:21.176220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24629f6-4cb3-4eb3-b5cd-7a515cd67a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:21.177307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24629f6-4cb3-4eb3-b5cd-7a515cd67a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:21.177741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24629f6-4cb3-4eb3-b5cd-7a515cd67a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:21.178214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24629f6-4cb3-4eb3-b5cd-7a515cd67a8c admin admin] PATH_INFO: `/v3/projects/9a2585b72cb444e88a1a3396ce24d559/users/887ebf2b02984ae483f26c947f6fead1/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:21.180940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24629f6-4cb3-4eb3-b5cd-7a515cd67a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a2585b72cb444e88a1a3396ce24d559, user_id=887ebf2b02984ae483f26c947f6fead1, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:21.184368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24629f6-4cb3-4eb3-b5cd-7a515cd67a8c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:21.193315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20e6ae-4719-4c81-9839-c26ba98b352b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:21.194704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20e6ae-4719-4c81-9839-c26ba98b352b service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:21.200798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 823/1632] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:21 2022] PUT /identity/v3/projects/9a2585b72cb444e88a1a3396ce24d559/users/887ebf2b02984ae483f26c947f6fead1/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:21.203593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24629f6-4cb3-4eb3-b5cd-7a515cd67a8c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:21.227598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 810/1633] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:21.228984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6c392-1c04-43a6-bafe-51ab83dbf1a4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:21.229280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6c392-1c04-43a6-bafe-51ab83dbf1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:21.229550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6c392-1c04-43a6-bafe-51ab83dbf1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:21.279560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-14b6c392-1c04-43a6-bafe-51ab83dbf1a4 None None] MFA Rules not processed for user `887ebf2b02984ae483f26c947f6fead1`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:21.281030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-14b6c392-1c04-43a6-bafe-51ab83dbf1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:21.337673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24629f6-4cb3-4eb3-b5cd-7a515cd67a8c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:21.348553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643d349-c385-44f8-b447-767d48575e11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:21.349233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643d349-c385-44f8-b447-767d48575e11 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:21.349521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643d349-c385-44f8-b447-767d48575e11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:21.349826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643d349-c385-44f8-b447-767d48575e11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:21.383670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 811/1634] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:21.466305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949ea2e-2a9c-4b1e-8b46-c767f5c0c0ba None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:21.466620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949ea2e-2a9c-4b1e-8b46-c767f5c0c0ba None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:21.466962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949ea2e-2a9c-4b1e-8b46-c767f5c0c0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:21.520538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643d349-c385-44f8-b447-767d48575e11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:21.523852 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7949ea2e-2a9c-4b1e-8b46-c767f5c0c0ba None None] MFA Rules not processed for user `4570168001784e9eab4711bf23afb47b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:21.526129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643d349-c385-44f8-b447-767d48575e11 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:21.528358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7949ea2e-2a9c-4b1e-8b46-c767f5c0c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:21.554886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 824/1635] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:21.644913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 812/1636] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:21.657116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8a065-3928-4e16-9f5c-f5dc75d90708 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:21.657391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8a065-3928-4e16-9f5c-f5dc75d90708 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:21.657804 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8a065-3928-4e16-9f5c-f5dc75d90708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:21.676426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949ea2e-2a9c-4b1e-8b46-c767f5c0c0ba None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:21.695065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-32f8a065-3928-4e16-9f5c-f5dc75d90708 None None] MFA Rules not processed for user `4570168001784e9eab4711bf23afb47b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:21.697834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-32f8a065-3928-4e16-9f5c-f5dc75d90708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:21.699111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949ea2e-2a9c-4b1e-8b46-c767f5c0c0ba None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:21.706404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a9764-4ac4-4b83-b9cf-32c6f9299b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:21.709186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a9764-4ac4-4b83-b9cf-32c6f9299b56 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:21.709575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a9764-4ac4-4b83-b9cf-32c6f9299b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:21.710530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a9764-4ac4-4b83-b9cf-32c6f9299b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:21.716533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 825/1637] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:21.728819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a9764-4ac4-4b83-b9cf-32c6f9299b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:21.730474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a9764-4ac4-4b83-b9cf-32c6f9299b56 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:21.742600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 813/1638] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:43:21.796392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8a065-3928-4e16-9f5c-f5dc75d90708 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:21.816320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8a065-3928-4e16-9f5c-f5dc75d90708 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:21.831840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e67e3-6bc9-4cb3-85fb-2c56ce632f06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:21.832652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e67e3-6bc9-4cb3-85fb-2c56ce632f06 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:21.832995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e67e3-6bc9-4cb3-85fb-2c56ce632f06 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:21.833339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e67e3-6bc9-4cb3-85fb-2c56ce632f06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:21.846139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e67e3-6bc9-4cb3-85fb-2c56ce632f06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:21.848454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e67e3-6bc9-4cb3-85fb-2c56ce632f06 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:21.887065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 826/1639] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:22.078685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a9764-4ac4-4b83-b9cf-32c6f9299b56 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:22.102740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a9764-4ac4-4b83-b9cf-32c6f9299b56 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:22.112524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f3e77-7e0a-4a20-aa0a-112c6d5406be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:22.113432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f3e77-7e0a-4a20-aa0a-112c6d5406be service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:22.113862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f3e77-7e0a-4a20-aa0a-112c6d5406be service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:22.114450 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f3e77-7e0a-4a20-aa0a-112c6d5406be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:22.249201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f3e77-7e0a-4a20-aa0a-112c6d5406be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:22.251152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f3e77-7e0a-4a20-aa0a-112c6d5406be service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:22.276700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 814/1640] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:22.280126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e67e3-6bc9-4cb3-85fb-2c56ce632f06 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:22.432843 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e67e3-6bc9-4cb3-85fb-2c56ce632f06 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:22.445192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:22.446108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:22.446396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:22.446673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] PATH_INFO: `/v3/users/ffc1b3f143494a4a91e81729b7bd0754` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:22.447950 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc1b3f143494a4a91e81729b7bd0754)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:22.449075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:43:22.449075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Apr 29 18:43:22.449075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: Apr 29 18:43:22.450321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:22.585127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc1b3f143494a4a91e81729b7bd0754'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:22.589782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc1b3f143494a4a91e81729b7bd0754'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:22.636031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc1b3f143494a4a91e81729b7bd0754'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:22.657782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc1b3f143494a4a91e81729b7bd0754'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:22.683729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-69cff809-1c6c-4e8b-ab25-1e18484fe5ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc1b3f143494a4a91e81729b7bd0754'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:22.704572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153706-401f-42fd-8590-3487b7ac0b1b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:22.704877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153706-401f-42fd-8590-3487b7ac0b1b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:22.705341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153706-401f-42fd-8590-3487b7ac0b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:22.714053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 827/1641] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:22 2022] DELETE /identity/v3/users/ffc1b3f143494a4a91e81729b7bd0754 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:22.737053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-db153706-401f-42fd-8590-3487b7ac0b1b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:22.737053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-db153706-401f-42fd-8590-3487b7ac0b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:22.738029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c26adb-9c2f-4203-9d82-f00c0652f64a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:22.738524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c26adb-9c2f-4203-9d82-f00c0652f64a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:22.738693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c26adb-9c2f-4203-9d82-f00c0652f64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:22.779786 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-85c26adb-9c2f-4203-9d82-f00c0652f64a None None] MFA Rules not processed for user `94716f9f6776435ba260230e02c39718`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:22.781747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-85c26adb-9c2f-4203-9d82-f00c0652f64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:22.895405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 815/1642] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:22.896173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 828/1643] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:22.900747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db153706-401f-42fd-8590-3487b7ac0b1b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:22.902272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-db153706-401f-42fd-8590-3487b7ac0b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:22.914422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c26adb-9c2f-4203-9d82-f00c0652f64a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:22.916218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-85c26adb-9c2f-4203-9d82-f00c0652f64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:22.932537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db153706-401f-42fd-8590-3487b7ac0b1b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:22.942966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cf2be-d87d-4214-ba21-df3080b7a645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:22.943653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cf2be-d87d-4214-ba21-df3080b7a645 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:22.943944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cf2be-d87d-4214-ba21-df3080b7a645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:22.944194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cf2be-d87d-4214-ba21-df3080b7a645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:22.956353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cf2be-d87d-4214-ba21-df3080b7a645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:22.958217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cf2be-d87d-4214-ba21-df3080b7a645 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:22.999984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 816/1644] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:22 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:23.000540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c26adb-9c2f-4203-9d82-f00c0652f64a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:23.003836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cf2be-d87d-4214-ba21-df3080b7a645 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:23.169232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5848bfc-b876-4627-8ac7-88d015422433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:23.176747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5848bfc-b876-4627-8ac7-88d015422433 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:23.177104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5848bfc-b876-4627-8ac7-88d015422433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:23.178763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5848bfc-b876-4627-8ac7-88d015422433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:23.251886 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cf2be-d87d-4214-ba21-df3080b7a645 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:23.278954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0467bf4-4003-403d-9539-25338495b657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:23.286969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0467bf4-4003-403d-9539-25338495b657 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:23.287249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0467bf4-4003-403d-9539-25338495b657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:23.288882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0467bf4-4003-403d-9539-25338495b657 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:23.289603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0467bf4-4003-403d-9539-25338495b657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:23.292017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0467bf4-4003-403d-9539-25338495b657 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:23.361422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 817/1645] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:23 2022] POST /identity/v3/users => generated 309 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:43:23.364574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d86639-67a2-4ff4-aeb8-d8998e99c22d None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:23.369166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d86639-67a2-4ff4-aeb8-d8998e99c22d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:23.369391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d86639-67a2-4ff4-aeb8-d8998e99c22d None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:23.370827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 818/1646] 10.222.0.50 () {54 vars in 912 bytes} [Fri Apr 29 18:43:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:43:23.379376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d86639-67a2-4ff4-aeb8-d8998e99c22d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:23.422696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d86639-67a2-4ff4-aeb8-d8998e99c22d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:23.438170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5848bfc-b876-4627-8ac7-88d015422433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:23.441523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5848bfc-b876-4627-8ac7-88d015422433 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:23.444396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9cd51-9620-4188-a884-f7d6cebcdf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:23.445351 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9cd51-9620-4188-a884-f7d6cebcdf5a admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:23.445784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9cd51-9620-4188-a884-f7d6cebcdf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:23.446259 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9cd51-9620-4188-a884-f7d6cebcdf5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:23.450681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9cd51-9620-4188-a884-f7d6cebcdf5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:23.455559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9cd51-9620-4188-a884-f7d6cebcdf5a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:23.472100 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 829/1647] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:23.473782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecb0e6-2de2-4697-874d-67f0bea1259c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:23.476994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecb0e6-2de2-4697-874d-67f0bea1259c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:23.477474 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecb0e6-2de2-4697-874d-67f0bea1259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:23.480864 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 819/1648] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:23.484663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9cd51-9620-4188-a884-f7d6cebcdf5a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:23.506511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9cd51-9620-4188-a884-f7d6cebcdf5a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:23.517858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9c0c7-0027-485b-9771-a17578c7a031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:23.525581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9c0c7-0027-485b-9771-a17578c7a031 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:23.527084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9c0c7-0027-485b-9771-a17578c7a031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:23.527545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9c0c7-0027-485b-9771-a17578c7a031 admin admin] PATH_INFO: `/v3/projects/8e7d1929ad1140338a8109af1682708e/users/acc740f3d2c04d4193939b9802d12ebc/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:23.558415 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-77ecb0e6-2de2-4697-874d-67f0bea1259c None None] MFA Rules not processed for user `07849b2925334763b119111ed52dd34c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:23.559897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-77ecb0e6-2de2-4697-874d-67f0bea1259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:23.563549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9c0c7-0027-485b-9771-a17578c7a031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7d1929ad1140338a8109af1682708e, user_id=acc740f3d2c04d4193939b9802d12ebc, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:23.566397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9c0c7-0027-485b-9771-a17578c7a031 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:23.569445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 830/1649] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:43:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:23.596416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecb0e6-2de2-4697-874d-67f0bea1259c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:23.603003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-77ecb0e6-2de2-4697-874d-67f0bea1259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:23.613650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 820/1650] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:23 2022] PUT /identity/v3/projects/8e7d1929ad1140338a8109af1682708e/users/acc740f3d2c04d4193939b9802d12ebc/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:23.615830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9c0c7-0027-485b-9771-a17578c7a031 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:23.630814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecb0e6-2de2-4697-874d-67f0bea1259c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:23.797513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca331413-2026-49c1-ad6f-52623f9c316b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:23.798183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca331413-2026-49c1-ad6f-52623f9c316b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:23.798462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca331413-2026-49c1-ad6f-52623f9c316b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:23.798733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca331413-2026-49c1-ad6f-52623f9c316b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:23.809492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca331413-2026-49c1-ad6f-52623f9c316b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:23.812473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca331413-2026-49c1-ad6f-52623f9c316b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:23.817841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9c0c7-0027-485b-9771-a17578c7a031 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:23.822590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 831/1651] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:43:23.830796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06034be-0bec-448e-9863-e66cf7ab5c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:23.832524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06034be-0bec-448e-9863-e66cf7ab5c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:23.832906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06034be-0bec-448e-9863-e66cf7ab5c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:23.833897 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06034be-0bec-448e-9863-e66cf7ab5c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:23.834839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06034be-0bec-448e-9863-e66cf7ab5c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:23.836739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06034be-0bec-448e-9863-e66cf7ab5c09 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:23.867116 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 821/1652] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:23.869941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca331413-2026-49c1-ad6f-52623f9c316b service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:23.886185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca331413-2026-49c1-ad6f-52623f9c316b service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:23.893407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a861f-7c03-4002-b80e-7cab3511c11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:23.894779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a861f-7c03-4002-b80e-7cab3511c11d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:23.895505 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a861f-7c03-4002-b80e-7cab3511c11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:23.895505 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a861f-7c03-4002-b80e-7cab3511c11d admin admin] PATH_INFO: `/v3/projects/8e7d1929ad1140338a8109af1682708e/users/acc740f3d2c04d4193939b9802d12ebc/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:23.898481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a861f-7c03-4002-b80e-7cab3511c11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7d1929ad1140338a8109af1682708e, user_id=acc740f3d2c04d4193939b9802d12ebc, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:23.904164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a861f-7c03-4002-b80e-7cab3511c11d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:23.918376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 832/1653] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:23 2022] PUT /identity/v3/projects/8e7d1929ad1140338a8109af1682708e/users/acc740f3d2c04d4193939b9802d12ebc/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:24.420323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06034be-0bec-448e-9863-e66cf7ab5c09 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:24.423359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a06034be-0bec-448e-9863-e66cf7ab5c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:24.440018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06034be-0bec-448e-9863-e66cf7ab5c09 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:24.554876 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:24.556026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:24.556399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:24.557287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] PATH_INFO: `/v3/projects/1775b09c31a0480a99ba335c2fddd9bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:24.558657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1775b09c31a0480a99ba335c2fddd9bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:24.560329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Apr 29 18:43:24.560329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Apr 29 18:43:24.560329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: Apr 29 18:43:24.562029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:24.685689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1775b09c31a0480a99ba335c2fddd9bc', 'request_id': 'req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:24.705861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1775b09c31a0480a99ba335c2fddd9bc', 'request_id': 'req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:24.727233 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1775b09c31a0480a99ba335c2fddd9bc', 'request_id': 'req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:24.752669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] The token cache is being invalidate because project 1775b09c31a0480a99ba335c2fddd9bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:24.753033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:24.753625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7bf2b2e8-88f8-414e-a100-3fd533dcb63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:24.758576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 822/1654] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:24 2022] DELETE /identity/v3/projects/1775b09c31a0480a99ba335c2fddd9bc => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:24.760737 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a11db-04ad-464a-b10a-90a860290454 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:24.761029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a11db-04ad-464a-b10a-90a860290454 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:24.761711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a11db-04ad-464a-b10a-90a860290454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:24.803008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-015a11db-04ad-464a-b10a-90a860290454 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:24.806648 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-015a11db-04ad-464a-b10a-90a860290454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:24.976467 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 833/1655] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:24.984584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75541f-1e4c-4030-b832-b0a48aee1555 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:24.984879 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75541f-1e4c-4030-b832-b0a48aee1555 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:24.985162 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75541f-1e4c-4030-b832-b0a48aee1555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.020741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-de75541f-1e4c-4030-b832-b0a48aee1555 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:25.022316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-de75541f-1e4c-4030-b832-b0a48aee1555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:25.045672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a11db-04ad-464a-b10a-90a860290454 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:25.046388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 823/1656] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:25.058404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a11db-04ad-464a-b10a-90a860290454 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:25.065544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25fe86-3a58-4c53-bead-dc73438fa3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:25.067244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25fe86-3a58-4c53-bead-dc73438fa3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:25.067810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25fe86-3a58-4c53-bead-dc73438fa3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:25.068295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25fe86-3a58-4c53-bead-dc73438fa3f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.069584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25fe86-3a58-4c53-bead-dc73438fa3f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:25.070166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25fe86-3a58-4c53-bead-dc73438fa3f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:25.073605 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25fe86-3a58-4c53-bead-dc73438fa3f2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:25.104267 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75541f-1e4c-4030-b832-b0a48aee1555 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:25.116345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 834/1657] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:25.119055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75541f-1e4c-4030-b832-b0a48aee1555 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:25.124404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdcdc0-d3f8-476b-9868-7ac76d3781be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:25.125237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdcdc0-d3f8-476b-9868-7ac76d3781be admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:25.125674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdcdc0-d3f8-476b-9868-7ac76d3781be admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:25.126183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdcdc0-d3f8-476b-9868-7ac76d3781be admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.127077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdcdc0-d3f8-476b-9868-7ac76d3781be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:25.130322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdcdc0-d3f8-476b-9868-7ac76d3781be admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:25.174031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25fe86-3a58-4c53-bead-dc73438fa3f2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:25.184720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 824/1658] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:25 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:25.189278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25fe86-3a58-4c53-bead-dc73438fa3f2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:25.335874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73017207-49e0-43dc-aeaa-90fcd859c34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:25.337833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73017207-49e0-43dc-aeaa-90fcd859c34a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:25.338854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73017207-49e0-43dc-aeaa-90fcd859c34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:25.340316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73017207-49e0-43dc-aeaa-90fcd859c34a admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.343706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73017207-49e0-43dc-aeaa-90fcd859c34a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:25.349393 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73017207-49e0-43dc-aeaa-90fcd859c34a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:25.427901 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdcdc0-d3f8-476b-9868-7ac76d3781be admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:25.449081 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 835/1659] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:25 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:25.452349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdcdc0-d3f8-476b-9868-7ac76d3781be admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:25.482480 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1f7fe-ced0-4b44-a865-851007d27edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:25.483778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1f7fe-ced0-4b44-a865-851007d27edb admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:25.484848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1f7fe-ced0-4b44-a865-851007d27edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:25.488049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1f7fe-ced0-4b44-a865-851007d27edb admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.489839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1f7fe-ced0-4b44-a865-851007d27edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:25.496389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1f7fe-ced0-4b44-a865-851007d27edb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:25.513610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081c7e0-fc40-4185-a277-747185329aa4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:25.514588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081c7e0-fc40-4185-a277-747185329aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:25.515538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081c7e0-fc40-4185-a277-747185329aa4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.517542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 836/1660] 10.222.0.50 () {54 vars in 960 bytes} [Fri Apr 29 18:43:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:43:25.529084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdedd6-9227-412f-8676-497c227dc5f0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:25.529687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdedd6-9227-412f-8676-497c227dc5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:25.530106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdedd6-9227-412f-8676-497c227dc5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.536583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 825/1661] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:25.545950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1f7fe-ced0-4b44-a865-851007d27edb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:25.575680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-15fdedd6-9227-412f-8676-497c227dc5f0 None None] MFA Rules not processed for user `6e934c8e7dd0425cac68eb91e5e2c0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:25.576975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-15fdedd6-9227-412f-8676-497c227dc5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:25.577543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1f7fe-ced0-4b44-a865-851007d27edb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:25.597236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d160c-3061-42de-847d-5acc53b034df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:25.599927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d160c-3061-42de-847d-5acc53b034df admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:25.600456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d160c-3061-42de-847d-5acc53b034df admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:25.601295 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d160c-3061-42de-847d-5acc53b034df admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.602286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d160c-3061-42de-847d-5acc53b034df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:25.604346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d160c-3061-42de-847d-5acc53b034df admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:25.631026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 826/1662] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:25.638408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d160c-3061-42de-847d-5acc53b034df admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:25.669280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d160c-3061-42de-847d-5acc53b034df admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:25.674963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 837/1663] 10.222.0.50 () {58 vars in 1057 bytes} [Fri Apr 29 18:43:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:43:25.687146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdedd6-9227-412f-8676-497c227dc5f0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:25.697777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0ee95-a259-4991-81b8-c31d573fc303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:25.698861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0ee95-a259-4991-81b8-c31d573fc303 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:25.699250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0ee95-a259-4991-81b8-c31d573fc303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:25.699632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0ee95-a259-4991-81b8-c31d573fc303 admin admin] PATH_INFO: `/v3/projects/ffc107ba70644002a4fd585e47c37e7f/users/841c42d63b5b4bf19bffabc116ddf5d8/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.710626 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdedd6-9227-412f-8676-497c227dc5f0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:25.739201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6714ec-3775-4c53-abb3-aaa8336f9ab0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e934c8e7dd0425cac68eb91e5e2c0ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:25.739925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6714ec-3775-4c53-abb3-aaa8336f9ab0 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:25.740213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6714ec-3775-4c53-abb3-aaa8336f9ab0 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:25.740851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6714ec-3775-4c53-abb3-aaa8336f9ab0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.741095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0ee95-a259-4991-81b8-c31d573fc303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc107ba70644002a4fd585e47c37e7f, user_id=841c42d63b5b4bf19bffabc116ddf5d8, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:25.745485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0ee95-a259-4991-81b8-c31d573fc303 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:25.747156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ea6714ec-3775-4c53-abb3-aaa8336f9ab0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:25.778619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6714ec-3775-4c53-abb3-aaa8336f9ab0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:25.780361 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6714ec-3775-4c53-abb3-aaa8336f9ab0 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:25.802031 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 827/1664] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:25 2022] PUT /identity/v3/projects/ffc107ba70644002a4fd585e47c37e7f/users/841c42d63b5b4bf19bffabc116ddf5d8/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:25.803727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689f547-7090-4279-8748-b71341755d06 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:25.804135 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689f547-7090-4279-8748-b71341755d06 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:25.804555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689f547-7090-4279-8748-b71341755d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:25.804681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 838/1665] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:25.874095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d689f547-7090-4279-8748-b71341755d06 None None] MFA Rules not processed for user `841c42d63b5b4bf19bffabc116ddf5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:25.878930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d689f547-7090-4279-8748-b71341755d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:25.989857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6714ec-3775-4c53-abb3-aaa8336f9ab0 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:25.991280 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ea6714ec-3775-4c53-abb3-aaa8336f9ab0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:26.045502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6714ec-3775-4c53-abb3-aaa8336f9ab0 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:26.132674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 828/1666] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:26.133825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689f547-7090-4279-8748-b71341755d06 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:26.135375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d689f547-7090-4279-8748-b71341755d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:26.365716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689f547-7090-4279-8748-b71341755d06 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:26.379923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:26.381364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:26.382084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:26.382574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] PATH_INFO: `/v3/users/bb050519bc3547378364d107f054bebf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:26.384625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb050519bc3547378364d107f054bebf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:26.391257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:26.445777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:26.447699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:26.448156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:26.448658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] PATH_INFO: `/v3/users/965be658862043b98d2411b85b35f81b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:26.461611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=965be658862043b98d2411b85b35f81b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:26.486651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:26.501026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb050519bc3547378364d107f054bebf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:26.503054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb050519bc3547378364d107f054bebf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:26.531324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb050519bc3547378364d107f054bebf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:26.551040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb050519bc3547378364d107f054bebf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:26.564077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb050519bc3547378364d107f054bebf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:26.574676 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 829/1667] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:26 2022] DELETE /identity/v3/users/bb050519bc3547378364d107f054bebf => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:26.575919 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:26.579374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:26.663662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '965be658862043b98d2411b85b35f81b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:26.663662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '965be658862043b98d2411b85b35f81b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:26.712535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '965be658862043b98d2411b85b35f81b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:26.737635 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '965be658862043b98d2411b85b35f81b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:26.759952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-06a1baa8-effe-406f-98e5-5725fd00bd4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '965be658862043b98d2411b85b35f81b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:26.783351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 839/1668] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:25 2022] DELETE /identity/v3/users/965be658862043b98d2411b85b35f81b => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:26.787206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82889650-cc99-468b-8e54-93ddad126f93 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:26.787503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82889650-cc99-468b-8e54-93ddad126f93 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:26.787778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82889650-cc99-468b-8e54-93ddad126f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:26.790481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fc891-8c46-42cd-84ae-85de725e39ab admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:26.849869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-82889650-cc99-468b-8e54-93ddad126f93 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:26.851038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-82889650-cc99-468b-8e54-93ddad126f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:26.960947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e36a8a-b35f-4353-b1fe-dd7e88c0e1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:26.962785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e36a8a-b35f-4353-b1fe-dd7e88c0e1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:26.963708 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e36a8a-b35f-4353-b1fe-dd7e88c0e1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:26.964127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e36a8a-b35f-4353-b1fe-dd7e88c0e1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:26.965003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e36a8a-b35f-4353-b1fe-dd7e88c0e1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:26.971320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e36a8a-b35f-4353-b1fe-dd7e88c0e1a1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:27.013348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 840/1669] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:27.013348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76654a7d-71ca-44f9-814a-f689bd26ac95 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:27.013348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76654a7d-71ca-44f9-814a-f689bd26ac95 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:27.013348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76654a7d-71ca-44f9-814a-f689bd26ac95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:27.034258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 830/1670] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:26 2022] POST /identity/v3/projects => generated 373 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:27.038008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e36a8a-b35f-4353-b1fe-dd7e88c0e1a1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:27.042248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e8e36a8a-b35f-4353-b1fe-dd7e88c0e1a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:27.070019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-76654a7d-71ca-44f9-814a-f689bd26ac95 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:27.071365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-76654a7d-71ca-44f9-814a-f689bd26ac95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:27.243008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e36a8a-b35f-4353-b1fe-dd7e88c0e1a1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:27.259373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff7b93-e869-45b1-ab7b-0bcd9370bbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:27.260090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff7b93-e869-45b1-ab7b-0bcd9370bbc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:27.260373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff7b93-e869-45b1-ab7b-0bcd9370bbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:27.260677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff7b93-e869-45b1-ab7b-0bcd9370bbc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:27.262419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff7b93-e869-45b1-ab7b-0bcd9370bbc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:27.266250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff7b93-e869-45b1-ab7b-0bcd9370bbc9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:27.270640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 841/1671] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:27.271549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76654a7d-71ca-44f9-814a-f689bd26ac95 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:27.274752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-76654a7d-71ca-44f9-814a-f689bd26ac95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:27.316849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 831/1672] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:27 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:27.318329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff7b93-e869-45b1-ab7b-0bcd9370bbc9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:27.341012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff7b93-e869-45b1-ab7b-0bcd9370bbc9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:27.351023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc88b6f-b5c1-4fe4-bd6c-58da8db6acb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:27.353589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc88b6f-b5c1-4fe4-bd6c-58da8db6acb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:27.353943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc88b6f-b5c1-4fe4-bd6c-58da8db6acb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:27.354285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc88b6f-b5c1-4fe4-bd6c-58da8db6acb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:27.356092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc88b6f-b5c1-4fe4-bd6c-58da8db6acb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:27.358312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc88b6f-b5c1-4fe4-bd6c-58da8db6acb9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:27.385420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 832/1673] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:27.391947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc88b6f-b5c1-4fe4-bd6c-58da8db6acb9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:27.395994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6cc88b6f-b5c1-4fe4-bd6c-58da8db6acb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:27.418319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76654a7d-71ca-44f9-814a-f689bd26ac95 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:27.425492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc88b6f-b5c1-4fe4-bd6c-58da8db6acb9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:27.429078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aa6c7-6e83-4bbd-8c63-b0a96be47ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:27.431146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aa6c7-6e83-4bbd-8c63-b0a96be47ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:27.432065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aa6c7-6e83-4bbd-8c63-b0a96be47ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:27.433681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aa6c7-6e83-4bbd-8c63-b0a96be47ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:27.436138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71017ad7-1d92-471d-a811-0c4ddf4aa570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:27.438875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71017ad7-1d92-471d-a811-0c4ddf4aa570 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:27.439467 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71017ad7-1d92-471d-a811-0c4ddf4aa570 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:27.440784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71017ad7-1d92-471d-a811-0c4ddf4aa570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:27.458456 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aa6c7-6e83-4bbd-8c63-b0a96be47ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:27.460604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aa6c7-6e83-4bbd-8c63-b0a96be47ad5 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:27.462959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71017ad7-1d92-471d-a811-0c4ddf4aa570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:27.466058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71017ad7-1d92-471d-a811-0c4ddf4aa570 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:27.486179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 842/1674] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:27.488024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aa6c7-6e83-4bbd-8c63-b0a96be47ad5 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:27.492785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 833/1675] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:27.505517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aa6c7-6e83-4bbd-8c63-b0a96be47ad5 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:27.518524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fcb92-dcd6-41c2-97fe-5aae4f1586c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:27.519724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fcb92-dcd6-41c2-97fe-5aae4f1586c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:27.520271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fcb92-dcd6-41c2-97fe-5aae4f1586c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:27.520758 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fcb92-dcd6-41c2-97fe-5aae4f1586c8 admin admin] PATH_INFO: `/v3/projects/7284f076fb334864b4aad41d2640b239/users/1aafa0f82dd44b84a8239717fd9a411e/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:27.546609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fcb92-dcd6-41c2-97fe-5aae4f1586c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7284f076fb334864b4aad41d2640b239, user_id=1aafa0f82dd44b84a8239717fd9a411e, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:27.553323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fcb92-dcd6-41c2-97fe-5aae4f1586c8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:27.585753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71017ad7-1d92-471d-a811-0c4ddf4aa570 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:27.593839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 843/1676] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:27 2022] PUT /identity/v3/projects/7284f076fb334864b4aad41d2640b239/users/1aafa0f82dd44b84a8239717fd9a411e/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:27.596379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71017ad7-1d92-471d-a811-0c4ddf4aa570 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:27.705910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff68bd4-38a5-40ac-8d60-900e6ee9389b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:27.706632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff68bd4-38a5-40ac-8d60-900e6ee9389b admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:27.706920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff68bd4-38a5-40ac-8d60-900e6ee9389b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:27.707214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff68bd4-38a5-40ac-8d60-900e6ee9389b admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:27.707972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff68bd4-38a5-40ac-8d60-900e6ee9389b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:27.709969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff68bd4-38a5-40ac-8d60-900e6ee9389b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:27.725999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fcb92-dcd6-41c2-97fe-5aae4f1586c8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:27.726437 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 834/1677] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:27.754545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fcb92-dcd6-41c2-97fe-5aae4f1586c8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:27.771634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba21e5-b4e7-49bd-bc35-41a50b6b8513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:27.772632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba21e5-b4e7-49bd-bc35-41a50b6b8513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:27.772981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba21e5-b4e7-49bd-bc35-41a50b6b8513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:27.777678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba21e5-b4e7-49bd-bc35-41a50b6b8513 admin admin] PATH_INFO: `/v3/projects/7284f076fb334864b4aad41d2640b239/users/1aafa0f82dd44b84a8239717fd9a411e/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:27.796356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba21e5-b4e7-49bd-bc35-41a50b6b8513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7284f076fb334864b4aad41d2640b239, user_id=1aafa0f82dd44b84a8239717fd9a411e, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:27.800289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba21e5-b4e7-49bd-bc35-41a50b6b8513 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:27.839903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289c29a-e872-4d43-8937-89a9d897b482 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:27.840198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289c29a-e872-4d43-8937-89a9d897b482 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:27.840478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289c29a-e872-4d43-8937-89a9d897b482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:27.841875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 844/1678] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:27 2022] PUT /identity/v3/projects/7284f076fb334864b4aad41d2640b239/users/1aafa0f82dd44b84a8239717fd9a411e/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:27.886371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1289c29a-e872-4d43-8937-89a9d897b482 None None] MFA Rules not processed for user `1aafa0f82dd44b84a8239717fd9a411e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:27.887603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1289c29a-e872-4d43-8937-89a9d897b482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:28.045069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 835/1679] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:28.055869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba21e5-b4e7-49bd-bc35-41a50b6b8513 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:28.059269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a1ba21e5-b4e7-49bd-bc35-41a50b6b8513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:28.191300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba21e5-b4e7-49bd-bc35-41a50b6b8513 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:28.204930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db7b7c-ffa0-4560-bf76-a3a585489dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:28.205834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db7b7c-ffa0-4560-bf76-a3a585489dbe service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:28.206338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db7b7c-ffa0-4560-bf76-a3a585489dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:28.206678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db7b7c-ffa0-4560-bf76-a3a585489dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:28.238242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db7b7c-ffa0-4560-bf76-a3a585489dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:28.239807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db7b7c-ffa0-4560-bf76-a3a585489dbe service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:28.264085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 845/1680] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:28.453820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289c29a-e872-4d43-8937-89a9d897b482 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:28.469633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289c29a-e872-4d43-8937-89a9d897b482 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:28.480456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ba8bd-dbad-457a-9d7a-43b323ef350b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:28.481260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba8bd-dbad-457a-9d7a-43b323ef350b service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:28.481654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba8bd-dbad-457a-9d7a-43b323ef350b service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:28.482124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba8bd-dbad-457a-9d7a-43b323ef350b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:28.485052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5e6ba8bd-dbad-457a-9d7a-43b323ef350b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:28.647088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ba8bd-dbad-457a-9d7a-43b323ef350b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:28.649405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ba8bd-dbad-457a-9d7a-43b323ef350b service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:28.673830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 836/1681] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:29.050617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db7b7c-ffa0-4560-bf76-a3a585489dbe service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:29.051606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a2db7b7c-ffa0-4560-bf76-a3a585489dbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:29.145898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95c1b-2cf9-4c9b-b085-649db9d9c9ee None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:29.146217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95c1b-2cf9-4c9b-b085-649db9d9c9ee None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:29.146553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95c1b-2cf9-4c9b-b085-649db9d9c9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:29.199908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-aaf95c1b-2cf9-4c9b-b085-649db9d9c9ee None None] MFA Rules not processed for user `acc740f3d2c04d4193939b9802d12ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:29.205385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-aaf95c1b-2cf9-4c9b-b085-649db9d9c9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:29.255112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db7b7c-ffa0-4560-bf76-a3a585489dbe service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:29.263619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:29.265502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:29.265785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:29.266466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] PATH_INFO: `/v3/projects/93c51c2ae9f342c3be4a8029be23f7a3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:29.271291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c51c2ae9f342c3be4a8029be23f7a3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:29.276921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:29.395995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 837/1682] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:43:29.435829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c51c2ae9f342c3be4a8029be23f7a3', 'request_id': 'req-f119cb30-2f9c-43e2-a36a-1839300d6005'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:29.456537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c51c2ae9f342c3be4a8029be23f7a3', 'request_id': 'req-f119cb30-2f9c-43e2-a36a-1839300d6005'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:29.471899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c51c2ae9f342c3be4a8029be23f7a3', 'request_id': 'req-f119cb30-2f9c-43e2-a36a-1839300d6005'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:29.490780 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] The token cache is being invalidate because project 93c51c2ae9f342c3be4a8029be23f7a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:29.491241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:29.494893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f119cb30-2f9c-43e2-a36a-1839300d6005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:29.502513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 846/1683] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:29 2022] DELETE /identity/v3/projects/93c51c2ae9f342c3be4a8029be23f7a3 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:29.508652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95c1b-2cf9-4c9b-b085-649db9d9c9ee None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:29.528618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95c1b-2cf9-4c9b-b085-649db9d9c9ee None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:29.593791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b5427-2950-418e-85e4-08e2a3f4af22 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:29.594418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b5427-2950-418e-85e4-08e2a3f4af22 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:29.594829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b5427-2950-418e-85e4-08e2a3f4af22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:29.617779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:29.618998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:29.619469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:29.619928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] PATH_INFO: `/v3/users/887ebf2b02984ae483f26c947f6fead1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:29.626461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] RBAC: Authorizing `identity:delete_user(user_id=887ebf2b02984ae483f26c947f6fead1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:29.628700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:29.634521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-429b5427-2950-418e-85e4-08e2a3f4af22 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:29.638247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-429b5427-2950-418e-85e4-08e2a3f4af22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:29.716447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '887ebf2b02984ae483f26c947f6fead1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:29.717245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '887ebf2b02984ae483f26c947f6fead1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:29.747045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '887ebf2b02984ae483f26c947f6fead1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:29.761462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '887ebf2b02984ae483f26c947f6fead1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:29.780993 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '887ebf2b02984ae483f26c947f6fead1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:29.789278 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 838/1684] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:29 2022] DELETE /identity/v3/users/887ebf2b02984ae483f26c947f6fead1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:29.794978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 847/1685] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:43:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:43:29.828745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:29.830221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:29.993987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e19ce9-c06b-414b-87f4-70b4be1da8de admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:30.008670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:30.009498 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:30.009778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:30.010050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] PATH_INFO: `/v3/projects/609e205e19d14496af1b90aa48697c44` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:30.012277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=609e205e19d14496af1b90aa48697c44)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:30.013577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:30.051199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b5427-2950-418e-85e4-08e2a3f4af22 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:30.056239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-429b5427-2950-418e-85e4-08e2a3f4af22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:30.150320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b5427-2950-418e-85e4-08e2a3f4af22 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:30.170234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609e205e19d14496af1b90aa48697c44', 'request_id': 'req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:30.194496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609e205e19d14496af1b90aa48697c44', 'request_id': 'req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:30.212352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609e205e19d14496af1b90aa48697c44', 'request_id': 'req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:30.259424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] The token cache is being invalidate because project 609e205e19d14496af1b90aa48697c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:30.260180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:30.261041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1b3e4b28-cd6c-46c0-8a07-6e3fce06bb16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:30.262799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 839/1686] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:29 2022] DELETE /identity/v3/projects/609e205e19d14496af1b90aa48697c44 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:30.270783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433bb9a-6c19-4ed3-bd22-900833c86501 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:30.271952 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433bb9a-6c19-4ed3-bd22-900833c86501 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:30.272468 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433bb9a-6c19-4ed3-bd22-900833c86501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:30.322251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0433bb9a-6c19-4ed3-bd22-900833c86501 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:30.327310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0433bb9a-6c19-4ed3-bd22-900833c86501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:30.482517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 840/1687] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:30.489741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e9dd2-3840-4278-92cf-399ffd63fe30 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:30.490114 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e9dd2-3840-4278-92cf-399ffd63fe30 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:30.490463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e9dd2-3840-4278-92cf-399ffd63fe30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:30.523846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-767e9dd2-3840-4278-92cf-399ffd63fe30 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:30.527422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-767e9dd2-3840-4278-92cf-399ffd63fe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:30.552346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 841/1688] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:43:30.556056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e9dd2-3840-4278-92cf-399ffd63fe30 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:30.574365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e9dd2-3840-4278-92cf-399ffd63fe30 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:30.589202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dc4ce-c2c3-499b-882f-1c27123c8f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:30.589921 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dc4ce-c2c3-499b-882f-1c27123c8f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:30.590201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dc4ce-c2c3-499b-882f-1c27123c8f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:30.590475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dc4ce-c2c3-499b-882f-1c27123c8f88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:30.591366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dc4ce-c2c3-499b-882f-1c27123c8f88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:30.591645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dc4ce-c2c3-499b-882f-1c27123c8f88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:30.598045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dc4ce-c2c3-499b-882f-1c27123c8f88 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:30.628917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f848dec-fdc5-43bf-921b-cabc37810a0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:30.630440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f848dec-fdc5-43bf-921b-cabc37810a0c service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:30.631045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f848dec-fdc5-43bf-921b-cabc37810a0c service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:30.631494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f848dec-fdc5-43bf-921b-cabc37810a0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:30.636798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 842/1689] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:43:30.637782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5f848dec-fdc5-43bf-921b-cabc37810a0c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:30.642973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dc4ce-c2c3-499b-882f-1c27123c8f88 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:30.666700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dc4ce-c2c3-499b-882f-1c27123c8f88 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:30.685369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5623f9-87ad-49d3-81c2-0622dc6d94fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:30.686156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5623f9-87ad-49d3-81c2-0622dc6d94fb admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:30.686517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5623f9-87ad-49d3-81c2-0622dc6d94fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:30.686905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5623f9-87ad-49d3-81c2-0622dc6d94fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:30.688205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5623f9-87ad-49d3-81c2-0622dc6d94fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:30.691420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5623f9-87ad-49d3-81c2-0622dc6d94fb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:30.743687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 843/1690] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:30 2022] POST /identity/v3/projects => generated 403 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:30.753537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5623f9-87ad-49d3-81c2-0622dc6d94fb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:30.814286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f848dec-fdc5-43bf-921b-cabc37810a0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:30.820331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f848dec-fdc5-43bf-921b-cabc37810a0c service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:30.847909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 848/1691] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:43:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 797 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Apr 29 18:43:30.878736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5623f9-87ad-49d3-81c2-0622dc6d94fb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:30.889053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbb1ab-c416-4d15-9d94-3126dadb0028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:30.890077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbb1ab-c416-4d15-9d94-3126dadb0028 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:30.890359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbb1ab-c416-4d15-9d94-3126dadb0028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:30.890628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbb1ab-c416-4d15-9d94-3126dadb0028 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:30.891328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbb1ab-c416-4d15-9d94-3126dadb0028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:30.893319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbb1ab-c416-4d15-9d94-3126dadb0028 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:30.945983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 844/1692] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:30 2022] POST /identity/v3/users => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:43:30.947047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f848dec-fdc5-43bf-921b-cabc37810a0c service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:30.969353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f848dec-fdc5-43bf-921b-cabc37810a0c service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:31.007410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8778f68-fc6c-4b38-b9fa-40fda8e2b46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:31.009634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8778f68-fc6c-4b38-b9fa-40fda8e2b46a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:31.009936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8778f68-fc6c-4b38-b9fa-40fda8e2b46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:31.010215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8778f68-fc6c-4b38-b9fa-40fda8e2b46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:31.017815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8778f68-fc6c-4b38-b9fa-40fda8e2b46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:31.029374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8778f68-fc6c-4b38-b9fa-40fda8e2b46a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:31.079945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 849/1693] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:43:31.081379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbb1ab-c416-4d15-9d94-3126dadb0028 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:31.102604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbb1ab-c416-4d15-9d94-3126dadb0028 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:31.122545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f77ca1-89d1-4f2e-bf6f-922846d0c9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:31.123262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f77ca1-89d1-4f2e-bf6f-922846d0c9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:31.128963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f77ca1-89d1-4f2e-bf6f-922846d0c9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:31.133026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f77ca1-89d1-4f2e-bf6f-922846d0c9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:31.135260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f77ca1-89d1-4f2e-bf6f-922846d0c9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:31.143014 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f77ca1-89d1-4f2e-bf6f-922846d0c9a4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:31.195176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 845/1694] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:31.196789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8778f68-fc6c-4b38-b9fa-40fda8e2b46a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:31.220394 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8778f68-fc6c-4b38-b9fa-40fda8e2b46a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:31.251436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83041d-560c-4a4b-aec9-039cfccf30bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:31.253141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83041d-560c-4a4b-aec9-039cfccf30bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:31.265771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83041d-560c-4a4b-aec9-039cfccf30bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:31.266181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83041d-560c-4a4b-aec9-039cfccf30bf admin admin] PATH_INFO: `/v3/projects/8707aae31fdc4f8d81446a45a3f80d34/users/2a21e656f0194f03b8ea6b42dbf71a9c/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:31.354011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83041d-560c-4a4b-aec9-039cfccf30bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8707aae31fdc4f8d81446a45a3f80d34, user_id=2a21e656f0194f03b8ea6b42dbf71a9c, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:31.368970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83041d-560c-4a4b-aec9-039cfccf30bf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:31.399690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 850/1695] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:31 2022] PUT /identity/v3/projects/8707aae31fdc4f8d81446a45a3f80d34/users/2a21e656f0194f03b8ea6b42dbf71a9c/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:43:31.412383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5782-9475-469c-8e03-6187e9a7d2ac None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:31.413285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5782-9475-469c-8e03-6187e9a7d2ac None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:31.413825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5782-9475-469c-8e03-6187e9a7d2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:31.477981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-53df5782-9475-469c-8e03-6187e9a7d2ac None None] MFA Rules not processed for user `2a21e656f0194f03b8ea6b42dbf71a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:31.479425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-53df5782-9475-469c-8e03-6187e9a7d2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:31.632388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 846/1696] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:31.636769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83041d-560c-4a4b-aec9-039cfccf30bf admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:31.804173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83041d-560c-4a4b-aec9-039cfccf30bf admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:31.820757 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540e06c-87b6-4791-9b7c-6b3b9c620539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:31.824050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540e06c-87b6-4791-9b7c-6b3b9c620539 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:31.824552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540e06c-87b6-4791-9b7c-6b3b9c620539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:31.825513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540e06c-87b6-4791-9b7c-6b3b9c620539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:31.827190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540e06c-87b6-4791-9b7c-6b3b9c620539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:31.834672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540e06c-87b6-4791-9b7c-6b3b9c620539 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:31.903902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 851/1697] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:31 2022] POST /identity/v3/projects => generated 405 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:43:31.909711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df5782-9475-469c-8e03-6187e9a7d2ac None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:31.928336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df5782-9475-469c-8e03-6187e9a7d2ac None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:32.013642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f59f70-9b35-4da8-9c91-235888073d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:32.014688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59f70-9b35-4da8-9c91-235888073d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:32.015125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59f70-9b35-4da8-9c91-235888073d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:32.015695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59f70-9b35-4da8-9c91-235888073d04 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:32.016832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f59f70-9b35-4da8-9c91-235888073d04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:32.020114 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f59f70-9b35-4da8-9c91-235888073d04 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:32.060601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 847/1698] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:31 2022] POST /identity/v3/users => generated 309 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:32.085790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540e06c-87b6-4791-9b7c-6b3b9c620539 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:32.099622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540e06c-87b6-4791-9b7c-6b3b9c620539 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:32.106293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262681d8-d888-472f-b779-3d28d4267190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:32.107923 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262681d8-d888-472f-b779-3d28d4267190 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:32.108357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262681d8-d888-472f-b779-3d28d4267190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:32.108828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262681d8-d888-472f-b779-3d28d4267190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:32.109703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262681d8-d888-472f-b779-3d28d4267190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:32.111773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262681d8-d888-472f-b779-3d28d4267190 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:32.149720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f59f70-9b35-4da8-9c91-235888073d04 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:32.187142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f59f70-9b35-4da8-9c91-235888073d04 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:32.187759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 852/1699] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:32.210713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c7915-84ac-4988-97a8-4da257e925f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:32.212263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c7915-84ac-4988-97a8-4da257e925f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:32.212263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c7915-84ac-4988-97a8-4da257e925f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:32.212263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c7915-84ac-4988-97a8-4da257e925f1 admin admin] PATH_INFO: `/v3/projects/3b2bec523ca847208dd2402fce26185e/users/40157b0a618641828c5e45d40e823173/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:32.258161 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c7915-84ac-4988-97a8-4da257e925f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2bec523ca847208dd2402fce26185e, user_id=40157b0a618641828c5e45d40e823173, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:32.260856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c7915-84ac-4988-97a8-4da257e925f1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:32.297679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 848/1700] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:32 2022] PUT /identity/v3/projects/3b2bec523ca847208dd2402fce26185e/users/40157b0a618641828c5e45d40e823173/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:32.298214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262681d8-d888-472f-b779-3d28d4267190 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:32.311999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262681d8-d888-472f-b779-3d28d4267190 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:32.430063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ba0bc-d474-4fae-be86-287bc3185597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:32.431227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ba0bc-d474-4fae-be86-287bc3185597 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:32.431792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ba0bc-d474-4fae-be86-287bc3185597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:32.432072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ba0bc-d474-4fae-be86-287bc3185597 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:32.433039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ba0bc-d474-4fae-be86-287bc3185597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:32.437774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ba0bc-d474-4fae-be86-287bc3185597 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:32.457643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c7915-84ac-4988-97a8-4da257e925f1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:32.458966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 853/1701] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:43:32.473488 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c7915-84ac-4988-97a8-4da257e925f1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:32.482914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d611ed-d0af-49d7-a1da-0c1c25f8df46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:32.491674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d611ed-d0af-49d7-a1da-0c1c25f8df46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:32.491674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d611ed-d0af-49d7-a1da-0c1c25f8df46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:32.491674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d611ed-d0af-49d7-a1da-0c1c25f8df46 admin admin] PATH_INFO: `/v3/projects/3b2bec523ca847208dd2402fce26185e/users/40157b0a618641828c5e45d40e823173/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:32.491674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d611ed-d0af-49d7-a1da-0c1c25f8df46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2bec523ca847208dd2402fce26185e, user_id=40157b0a618641828c5e45d40e823173, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:32.494018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d611ed-d0af-49d7-a1da-0c1c25f8df46 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:32.524675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a576ede-595d-4a21-82ea-8ccb6832d3cf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:32.525010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a576ede-595d-4a21-82ea-8ccb6832d3cf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:32.525297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a576ede-595d-4a21-82ea-8ccb6832d3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:32.527741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 849/1702] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:32 2022] PUT /identity/v3/projects/3b2bec523ca847208dd2402fce26185e/users/40157b0a618641828c5e45d40e823173/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:32.571962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0a576ede-595d-4a21-82ea-8ccb6832d3cf None None] MFA Rules not processed for user `40157b0a618641828c5e45d40e823173`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:32.580951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0a576ede-595d-4a21-82ea-8ccb6832d3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:32.599903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6f4ee-4760-4bb8-ad72-5ba9dc042da7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:32.600196 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6f4ee-4760-4bb8-ad72-5ba9dc042da7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:32.600474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6f4ee-4760-4bb8-ad72-5ba9dc042da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:32.902658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 854/1703] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:32 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:32.904391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a576ede-595d-4a21-82ea-8ccb6832d3cf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:32.907555 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0a576ede-595d-4a21-82ea-8ccb6832d3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:33.033159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a576ede-595d-4a21-82ea-8ccb6832d3cf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:33.044924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:33.045333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f7e6f4ee-4760-4bb8-ad72-5ba9dc042da7 None None] MFA Rules not processed for user `64c8eec4efab42bfa5d9e9b157883dd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:33.045994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:33.046327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:33.046675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] PATH_INFO: `/v3/projects/9a2585b72cb444e88a1a3396ce24d559` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:33.046787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f7e6f4ee-4760-4bb8-ad72-5ba9dc042da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:33.049018 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a2585b72cb444e88a1a3396ce24d559)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:33.051030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:33.147008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 850/1704] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 548 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:33.148948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6f4ee-4760-4bb8-ad72-5ba9dc042da7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:33.149590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a2585b72cb444e88a1a3396ce24d559', 'request_id': 'req-416d1543-63d6-4b3d-99df-dd990a43cc44'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:33.158214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f7e6f4ee-4760-4bb8-ad72-5ba9dc042da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:33.167131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a2585b72cb444e88a1a3396ce24d559', 'request_id': 'req-416d1543-63d6-4b3d-99df-dd990a43cc44'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:33.184874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a2585b72cb444e88a1a3396ce24d559', 'request_id': 'req-416d1543-63d6-4b3d-99df-dd990a43cc44'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:33.215099 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] The token cache is being invalidate because project 9a2585b72cb444e88a1a3396ce24d559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:33.215437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:33.217396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:33.222473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 855/1705] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:32 2022] DELETE /identity/v3/projects/9a2585b72cb444e88a1a3396ce24d559 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:43:33.222473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:33.223844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:33.257250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6f4ee-4760-4bb8-ad72-5ba9dc042da7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:33.350871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d136f22-73e0-43c6-8995-91037872b7c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:33.352337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d136f22-73e0-43c6-8995-91037872b7c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:33.352692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d136f22-73e0-43c6-8995-91037872b7c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:33.353108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d136f22-73e0-43c6-8995-91037872b7c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:33.361063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3d136f22-73e0-43c6-8995-91037872b7c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:33.390424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d1543-63d6-4b3d-99df-dd990a43cc44 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:33.396466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b97510-252f-49ac-8272-eece7b804f7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:33.397203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b97510-252f-49ac-8272-eece7b804f7a service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:33.397480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b97510-252f-49ac-8272-eece7b804f7a service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:33.397829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b97510-252f-49ac-8272-eece7b804f7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:33.403132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-92b97510-252f-49ac-8272-eece7b804f7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:33.548527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b97510-252f-49ac-8272-eece7b804f7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:33.550385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b97510-252f-49ac-8272-eece7b804f7a service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:33.559168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d136f22-73e0-43c6-8995-91037872b7c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:33.561576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d136f22-73e0-43c6-8995-91037872b7c4 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:33.571362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 856/1706] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:33.572835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56786459-bbb2-42d2-918f-a311e651449b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:33.573245 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56786459-bbb2-42d2-918f-a311e651449b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:33.573518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56786459-bbb2-42d2-918f-a311e651449b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:33.589263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 851/1707] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:33 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:33.638743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-56786459-bbb2-42d2-918f-a311e651449b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:33.640541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-56786459-bbb2-42d2-918f-a311e651449b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:33.851415 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 857/1708] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:33.853612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ada41-8650-41d8-9da7-caff249f0b80 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:33.853828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ada41-8650-41d8-9da7-caff249f0b80 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:33.854116 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ada41-8650-41d8-9da7-caff249f0b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:33.917793 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-235ada41-8650-41d8-9da7-caff249f0b80 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:33.930040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-235ada41-8650-41d8-9da7-caff249f0b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:33.947618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 852/1709] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:33.952327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56786459-bbb2-42d2-918f-a311e651449b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:33.963441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56786459-bbb2-42d2-918f-a311e651449b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:33.990040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181828-085f-4c33-a630-1680b8782edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:33.992928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181828-085f-4c33-a630-1680b8782edd admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:33.993957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181828-085f-4c33-a630-1680b8782edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:33.995016 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181828-085f-4c33-a630-1680b8782edd admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:33.997329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83181828-085f-4c33-a630-1680b8782edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:34.002305 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83181828-085f-4c33-a630-1680b8782edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:34.006608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83181828-085f-4c33-a630-1680b8782edd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:34.062625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ada41-8650-41d8-9da7-caff249f0b80 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:34.071452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 858/1710] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:34.080227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ada41-8650-41d8-9da7-caff249f0b80 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:34.120869 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7238822-f7e6-4a38-88fc-08790b19553a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:34.122154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7238822-f7e6-4a38-88fc-08790b19553a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:34.122442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7238822-f7e6-4a38-88fc-08790b19553a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:34.122720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7238822-f7e6-4a38-88fc-08790b19553a admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:34.127546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7238822-f7e6-4a38-88fc-08790b19553a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:34.132183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7238822-f7e6-4a38-88fc-08790b19553a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:34.236752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 853/1711] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:34 2022] POST /identity/v3/projects => generated 409 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:34.240996 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181828-085f-4c33-a630-1680b8782edd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:34.256924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181828-085f-4c33-a630-1680b8782edd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:34.410812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf3552-342e-4f85-a2fe-7ab4764c6321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:34.411885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf3552-342e-4f85-a2fe-7ab4764c6321 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:34.412374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf3552-342e-4f85-a2fe-7ab4764c6321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:34.412883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf3552-342e-4f85-a2fe-7ab4764c6321 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:34.413828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf3552-342e-4f85-a2fe-7ab4764c6321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:34.415970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf3552-342e-4f85-a2fe-7ab4764c6321 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:34.459469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7238822-f7e6-4a38-88fc-08790b19553a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:34.473301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 859/1712] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:34 2022] POST /identity/v3/users => generated 305 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:34.476639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7238822-f7e6-4a38-88fc-08790b19553a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:34.483234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3f9ef-dffc-4e6a-b6f1-ed91fdf2ee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:34.485474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3f9ef-dffc-4e6a-b6f1-ed91fdf2ee2e admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:34.485765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3f9ef-dffc-4e6a-b6f1-ed91fdf2ee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:34.486208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3f9ef-dffc-4e6a-b6f1-ed91fdf2ee2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:34.486966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3f9ef-dffc-4e6a-b6f1-ed91fdf2ee2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:34.488682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3f9ef-dffc-4e6a-b6f1-ed91fdf2ee2e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:34.513895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf3552-342e-4f85-a2fe-7ab4764c6321 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:34.525979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 854/1713] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:34.528190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf3552-342e-4f85-a2fe-7ab4764c6321 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:34.537307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1ed32-3e66-45b2-80a4-042e35b8dec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:34.538265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1ed32-3e66-45b2-80a4-042e35b8dec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:34.538547 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1ed32-3e66-45b2-80a4-042e35b8dec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:34.538817 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1ed32-3e66-45b2-80a4-042e35b8dec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:34.539551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1ed32-3e66-45b2-80a4-042e35b8dec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:34.541258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1ed32-3e66-45b2-80a4-042e35b8dec4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:34.566149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3f9ef-dffc-4e6a-b6f1-ed91fdf2ee2e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:34.579634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 860/1714] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:34.579941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3f9ef-dffc-4e6a-b6f1-ed91fdf2ee2e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:34.589499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2913d89-c488-45ba-8066-6360d0345a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:34.590737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2913d89-c488-45ba-8066-6360d0345a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:34.590876 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2913d89-c488-45ba-8066-6360d0345a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:34.591123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2913d89-c488-45ba-8066-6360d0345a2b admin admin] PATH_INFO: `/v3/projects/3b3df1ea2ae24c59901801dd9dae6ae0/users/7a26abe32aee4386a9ea9e850f6a83a9/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:34.616813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2913d89-c488-45ba-8066-6360d0345a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3df1ea2ae24c59901801dd9dae6ae0, user_id=7a26abe32aee4386a9ea9e850f6a83a9, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:34.621130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2913d89-c488-45ba-8066-6360d0345a2b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:34.667346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48373188-7e3c-46a5-b627-516b70137b00 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:34.667523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48373188-7e3c-46a5-b627-516b70137b00 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:34.667798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48373188-7e3c-46a5-b627-516b70137b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:34.670593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 855/1715] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:34 2022] PUT /identity/v3/projects/3b3df1ea2ae24c59901801dd9dae6ae0/users/7a26abe32aee4386a9ea9e850f6a83a9/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:34.709975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-48373188-7e3c-46a5-b627-516b70137b00 None None] MFA Rules not processed for user `7a26abe32aee4386a9ea9e850f6a83a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:34.711194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-48373188-7e3c-46a5-b627-516b70137b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:34.823057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 861/1716] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:34.830292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2913d89-c488-45ba-8066-6360d0345a2b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:34.916346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2913d89-c488-45ba-8066-6360d0345a2b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:34.943397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486d0b3-9d6e-448d-9e70-f44f16712530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:34.944371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486d0b3-9d6e-448d-9e70-f44f16712530 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:34.944769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486d0b3-9d6e-448d-9e70-f44f16712530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:34.945276 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486d0b3-9d6e-448d-9e70-f44f16712530 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:34.946394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486d0b3-9d6e-448d-9e70-f44f16712530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:34.949573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486d0b3-9d6e-448d-9e70-f44f16712530 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:35.000011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48373188-7e3c-46a5-b627-516b70137b00 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:35.003339 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 856/1717] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:34 2022] POST /identity/v3/projects => generated 409 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:35.017828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48373188-7e3c-46a5-b627-516b70137b00 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:35.098212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9a224-c94b-4f55-8b45-f95ea91dd002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:35.099471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9a224-c94b-4f55-8b45-f95ea91dd002 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:35.099811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9a224-c94b-4f55-8b45-f95ea91dd002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:35.100085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9a224-c94b-4f55-8b45-f95ea91dd002 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:35.101105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9a224-c94b-4f55-8b45-f95ea91dd002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:35.108358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9a224-c94b-4f55-8b45-f95ea91dd002 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:35.151342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486d0b3-9d6e-448d-9e70-f44f16712530 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:35.165731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 862/1718] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:34 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:35.168927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486d0b3-9d6e-448d-9e70-f44f16712530 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:35.181971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d0588-ec13-4398-b6ff-92ecd4540d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:35.182731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d0588-ec13-4398-b6ff-92ecd4540d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:35.184175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d0588-ec13-4398-b6ff-92ecd4540d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:35.184990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d0588-ec13-4398-b6ff-92ecd4540d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:35.187624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d0588-ec13-4398-b6ff-92ecd4540d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:35.192131 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d0588-ec13-4398-b6ff-92ecd4540d73 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:35.225309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9a224-c94b-4f55-8b45-f95ea91dd002 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:35.238818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 857/1719] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:43:35.239752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9a224-c94b-4f55-8b45-f95ea91dd002 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:35.256171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47a3bf-24d2-4f70-a30f-728a02980def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:35.257439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47a3bf-24d2-4f70-a30f-728a02980def admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:35.257868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47a3bf-24d2-4f70-a30f-728a02980def admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:35.258283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47a3bf-24d2-4f70-a30f-728a02980def admin admin] PATH_INFO: `/v3/projects/46dbb7100226420fbbf6a8f17ac4acce/users/2d688d6d8b63472a8512f0811efae197/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:35.304664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47a3bf-24d2-4f70-a30f-728a02980def admin admin] RBAC: Authorizing `identity:create_grant(project_id=46dbb7100226420fbbf6a8f17ac4acce, user_id=2d688d6d8b63472a8512f0811efae197, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:35.307385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47a3bf-24d2-4f70-a30f-728a02980def admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:35.343137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 863/1720] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:35 2022] PUT /identity/v3/projects/46dbb7100226420fbbf6a8f17ac4acce/users/2d688d6d8b63472a8512f0811efae197/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:35.347293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d0588-ec13-4398-b6ff-92ecd4540d73 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:35.361691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d0588-ec13-4398-b6ff-92ecd4540d73 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:35.517489 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea873e-9aa1-4aeb-8db1-bc98486f6199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:35.521673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea873e-9aa1-4aeb-8db1-bc98486f6199 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:35.521977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea873e-9aa1-4aeb-8db1-bc98486f6199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:35.522263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea873e-9aa1-4aeb-8db1-bc98486f6199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:35.525721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea873e-9aa1-4aeb-8db1-bc98486f6199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:35.525721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea873e-9aa1-4aeb-8db1-bc98486f6199 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:35.543020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 858/1721] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:35.543983 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47a3bf-24d2-4f70-a30f-728a02980def admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:35.560773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47a3bf-24d2-4f70-a30f-728a02980def admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:35.572904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a1177-d9b8-4264-9923-8e280f2557fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:35.573940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a1177-d9b8-4264-9923-8e280f2557fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:35.574104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a1177-d9b8-4264-9923-8e280f2557fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:35.574380 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a1177-d9b8-4264-9923-8e280f2557fb admin admin] PATH_INFO: `/v3/projects/46dbb7100226420fbbf6a8f17ac4acce/users/2d688d6d8b63472a8512f0811efae197/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:35.584657 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a1177-d9b8-4264-9923-8e280f2557fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=46dbb7100226420fbbf6a8f17ac4acce, user_id=2d688d6d8b63472a8512f0811efae197, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:35.587596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a1177-d9b8-4264-9923-8e280f2557fb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:35.619439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 864/1722] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:35 2022] PUT /identity/v3/projects/46dbb7100226420fbbf6a8f17ac4acce/users/2d688d6d8b63472a8512f0811efae197/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:43:35.621334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef345f3-24ba-4ca8-9773-f8aaf4579aeb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:35.621629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef345f3-24ba-4ca8-9773-f8aaf4579aeb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:35.621905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef345f3-24ba-4ca8-9773-f8aaf4579aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:35.679778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0ef345f3-24ba-4ca8-9773-f8aaf4579aeb None None] MFA Rules not processed for user `2d688d6d8b63472a8512f0811efae197`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:35.681070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0ef345f3-24ba-4ca8-9773-f8aaf4579aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:35.788182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 859/1723] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:35 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:35.804696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a1177-d9b8-4264-9923-8e280f2557fb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:35.809128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-131a1177-d9b8-4264-9923-8e280f2557fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:35.860770 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef345f3-24ba-4ca8-9773-f8aaf4579aeb None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:35.862560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0ef345f3-24ba-4ca8-9773-f8aaf4579aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:35.970301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a1177-d9b8-4264-9923-8e280f2557fb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:35.975188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c15cb-3f0c-44d5-9fc2-d56af3bc08cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:35.975917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c15cb-3f0c-44d5-9fc2-d56af3bc08cd service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:35.976200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c15cb-3f0c-44d5-9fc2-d56af3bc08cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:35.976475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c15cb-3f0c-44d5-9fc2-d56af3bc08cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:35.986506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef345f3-24ba-4ca8-9773-f8aaf4579aeb None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:35.989243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c15cb-3f0c-44d5-9fc2-d56af3bc08cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:35.992905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c15cb-3f0c-44d5-9fc2-d56af3bc08cd service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:35.997615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:35.998898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:35.999514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:36.000266 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] PATH_INFO: `/v3/users/94716f9f6776435ba260230e02c39718` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:36.001821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94716f9f6776435ba260230e02c39718)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:36.005976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 865/1724] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:35 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:36.006490 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:36.098042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94716f9f6776435ba260230e02c39718'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:36.101149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94716f9f6776435ba260230e02c39718'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:36.158475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94716f9f6776435ba260230e02c39718'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:36.175997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94716f9f6776435ba260230e02c39718'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:36.185563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94716f9f6776435ba260230e02c39718'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:36.194050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 860/1725] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:35 2022] DELETE /identity/v3/users/94716f9f6776435ba260230e02c39718 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:43:36.445396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c15cb-3f0c-44d5-9fc2-d56af3bc08cd service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:36.455026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c15cb-3f0c-44d5-9fc2-d56af3bc08cd service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:36.608615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef5bcc-5a9c-4a7b-99cc-5568d77405fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:36.610456 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef5bcc-5a9c-4a7b-99cc-5568d77405fb service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:36.613489 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef5bcc-5a9c-4a7b-99cc-5568d77405fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:36.613489 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef5bcc-5a9c-4a7b-99cc-5568d77405fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:36.629009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-80ef5bcc-5a9c-4a7b-99cc-5568d77405fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:36.769996 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef5bcc-5a9c-4a7b-99cc-5568d77405fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:36.771720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef5bcc-5a9c-4a7b-99cc-5568d77405fb service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:36.796107 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 866/1726] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:36 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:37.749888 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:37.751739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:37.926312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aecc50-ab77-4233-909b-858a8763ffc7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:37.944669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:37.945578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:37.945857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:37.946262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] PATH_INFO: `/v3/projects/5c9c70cdc1a247a2b79561a4b202a277` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:37.947505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c9c70cdc1a247a2b79561a4b202a277)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:37.949457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:38.056372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c9c70cdc1a247a2b79561a4b202a277', 'request_id': 'req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.083650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c9c70cdc1a247a2b79561a4b202a277', 'request_id': 'req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.099408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c9c70cdc1a247a2b79561a4b202a277', 'request_id': 'req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.136773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] The token cache is being invalidate because project 5c9c70cdc1a247a2b79561a4b202a277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:38.138254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.139035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.143063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 861/1727] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:37 2022] DELETE /identity/v3/projects/5c9c70cdc1a247a2b79561a4b202a277 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:38.147263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef5bcc-5a9c-4a7b-99cc-5568d77405fb service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:38.162882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef5bcc-5a9c-4a7b-99cc-5568d77405fb service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:38.309956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:38.317188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:38.361117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:38.362043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:38.362308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:38.362721 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] PATH_INFO: `/v3/users/acc740f3d2c04d4193939b9802d12ebc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:38.371570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc740f3d2c04d4193939b9802d12ebc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:38.380616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:38.562163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc740f3d2c04d4193939b9802d12ebc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.564224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc740f3d2c04d4193939b9802d12ebc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.568568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab7084-d7cc-423d-9878-8a9a51cacbb7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:38.609857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:38.618610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:38.619820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:38.623758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] PATH_INFO: `/v3/users/2a21e656f0194f03b8ea6b42dbf71a9c` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:38.630520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a21e656f0194f03b8ea6b42dbf71a9c)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:38.636180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc740f3d2c04d4193939b9802d12ebc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.642214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:38.673434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc740f3d2c04d4193939b9802d12ebc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.709517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d454e3ad-730b-49e8-bad2-6bf41195c9c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc740f3d2c04d4193939b9802d12ebc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.742100 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 867/1728] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:38 2022] DELETE /identity/v3/users/acc740f3d2c04d4193939b9802d12ebc => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:38.743893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8cacc-65b6-470d-9c48-3d4aa9d6452e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:38.746545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8cacc-65b6-470d-9c48-3d4aa9d6452e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:38.747093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8cacc-65b6-470d-9c48-3d4aa9d6452e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:38.836723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a21e656f0194f03b8ea6b42dbf71a9c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.838201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5fe8cacc-65b6-470d-9c48-3d4aa9d6452e None None] MFA Rules not processed for user `49aaea67fba64d2583d2023505ce7779`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:38.840167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a21e656f0194f03b8ea6b42dbf71a9c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.843125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5fe8cacc-65b6-470d-9c48-3d4aa9d6452e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:38.889129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a21e656f0194f03b8ea6b42dbf71a9c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.909516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a21e656f0194f03b8ea6b42dbf71a9c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.929826 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-987cbddb-a7fe-4aa5-85eb-17f2a7ee62b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a21e656f0194f03b8ea6b42dbf71a9c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:38.943764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 862/1729] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:38 2022] DELETE /identity/v3/users/2a21e656f0194f03b8ea6b42dbf71a9c => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:38.949224 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b92947-287d-46c7-b9c8-2763d9ba2e60 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:38.949902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b92947-287d-46c7-b9c8-2763d9ba2e60 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:38.950348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b92947-287d-46c7-b9c8-2763d9ba2e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:38.985614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d3b92947-287d-46c7-b9c8-2763d9ba2e60 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:38.987511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d3b92947-287d-46c7-b9c8-2763d9ba2e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:39.043327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 868/1730] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:39.050696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8cacc-65b6-470d-9c48-3d4aa9d6452e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:39.052523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5fe8cacc-65b6-470d-9c48-3d4aa9d6452e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:39.070051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 863/1731] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:39.103392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b92947-287d-46c7-b9c8-2763d9ba2e60 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:39.105375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d3b92947-287d-46c7-b9c8-2763d9ba2e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:39.199822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8cacc-65b6-470d-9c48-3d4aa9d6452e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:39.228602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b92947-287d-46c7-b9c8-2763d9ba2e60 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:39.234612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f2f94-c63b-436d-b788-0f9401cff5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:39.235955 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f2f94-c63b-436d-b788-0f9401cff5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:39.237385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f2f94-c63b-436d-b788-0f9401cff5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:39.237715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f2f94-c63b-436d-b788-0f9401cff5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:39.251625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f2f94-c63b-436d-b788-0f9401cff5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:39.253538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f2f94-c63b-436d-b788-0f9401cff5a7 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:39.267457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 864/1732] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:43:39.311238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fc35e-96ab-47b9-895d-009536e39e27 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:39.311570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fc35e-96ab-47b9-895d-009536e39e27 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:39.311803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fc35e-96ab-47b9-895d-009536e39e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:39.340659 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7b9ac-0feb-4463-a5c0-fffb9bae8623 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:39.341827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7b9ac-0feb-4463-a5c0-fffb9bae8623 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:39.342199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7b9ac-0feb-4463-a5c0-fffb9bae8623 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:39.342865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7b9ac-0feb-4463-a5c0-fffb9bae8623 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:39.362166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-433fc35e-96ab-47b9-895d-009536e39e27 None None] MFA Rules not processed for user `e9a2443e71c54075823f75fdb8df469e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:39.363379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-433fc35e-96ab-47b9-895d-009536e39e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:39.474985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7b9ac-0feb-4463-a5c0-fffb9bae8623 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:39.476764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7b9ac-0feb-4463-a5c0-fffb9bae8623 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:39.497375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 869/1733] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:43:39.513221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d34d4-6209-469d-9629-dcb1a5b0a553 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:39.513517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d34d4-6209-469d-9629-dcb1a5b0a553 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:39.513788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d34d4-6209-469d-9629-dcb1a5b0a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:39.516199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 865/1734] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:43:39.535018 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-135d34d4-6209-469d-9629-dcb1a5b0a553 None None] MFA Rules not processed for user `e9a2443e71c54075823f75fdb8df469e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:39.536446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-135d34d4-6209-469d-9629-dcb1a5b0a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:39.550363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 870/1735] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:39.619601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fc35e-96ab-47b9-895d-009536e39e27 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:39.630424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-433fc35e-96ab-47b9-895d-009536e39e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:39.679306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fc35e-96ab-47b9-895d-009536e39e27 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:39.709086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83222b44-ad4b-4da8-8013-11b078c7d0a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:39.710938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83222b44-ad4b-4da8-8013-11b078c7d0a1 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:39.711322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83222b44-ad4b-4da8-8013-11b078c7d0a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:39.711686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83222b44-ad4b-4da8-8013-11b078c7d0a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:39.734783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83222b44-ad4b-4da8-8013-11b078c7d0a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:39.736900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83222b44-ad4b-4da8-8013-11b078c7d0a1 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:39.770003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 866/1736] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:43:40.070707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d34d4-6209-469d-9629-dcb1a5b0a553 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:40.072162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-135d34d4-6209-469d-9629-dcb1a5b0a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:40.101837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d34d4-6209-469d-9629-dcb1a5b0a553 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:40.119714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:40.120653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:40.121146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:40.121238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] PATH_INFO: `/v3/projects/8e7d1929ad1140338a8109af1682708e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:40.124270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e7d1929ad1140338a8109af1682708e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:40.126305 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:40.248295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e7d1929ad1140338a8109af1682708e', 'request_id': 'req-cd628a7e-bf2f-405d-ae50-212478d042da'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:40.263913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e7d1929ad1140338a8109af1682708e', 'request_id': 'req-cd628a7e-bf2f-405d-ae50-212478d042da'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:40.289398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e7d1929ad1140338a8109af1682708e', 'request_id': 'req-cd628a7e-bf2f-405d-ae50-212478d042da'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:40.320981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] The token cache is being invalidate because project 8e7d1929ad1140338a8109af1682708e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:40.322669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:40.323876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:40.332698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 871/1737] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:40 2022] DELETE /identity/v3/projects/8e7d1929ad1140338a8109af1682708e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:40.337214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0ca54-20f2-4148-b403-95fa3bb0fe88 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:40.337772 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0ca54-20f2-4148-b403-95fa3bb0fe88 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:40.337997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0ca54-20f2-4148-b403-95fa3bb0fe88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:40.358579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:40.362701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:40.419439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-19c0ca54-20f2-4148-b403-95fa3bb0fe88 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:40.420752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-19c0ca54-20f2-4148-b403-95fa3bb0fe88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:40.563335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd628a7e-bf2f-405d-ae50-212478d042da admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:40.597384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:40.598695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:40.601164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:40.601703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] PATH_INFO: `/v3/projects/8707aae31fdc4f8d81446a45a3f80d34` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:40.607757 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8707aae31fdc4f8d81446a45a3f80d34)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:40.612410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:40.722350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 867/1738] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:40.724108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ffdad-dad9-49c5-9e42-3a389aa3f0da None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:40.724423 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ffdad-dad9-49c5-9e42-3a389aa3f0da None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:40.724766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ffdad-dad9-49c5-9e42-3a389aa3f0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:40.766035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-888ffdad-dad9-49c5-9e42-3a389aa3f0da None None] MFA Rules not processed for user `07849b2925334763b119111ed52dd34c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:40.769501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8707aae31fdc4f8d81446a45a3f80d34', 'request_id': 'req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:40.775283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-888ffdad-dad9-49c5-9e42-3a389aa3f0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:40.787055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8707aae31fdc4f8d81446a45a3f80d34', 'request_id': 'req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:40.833747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8707aae31fdc4f8d81446a45a3f80d34', 'request_id': 'req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:40.890316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] The token cache is being invalidate because project 8707aae31fdc4f8d81446a45a3f80d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:40.896759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:40.899128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4598db7-e2d9-4264-ac66-0b7a0dcf8906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:40.905490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 872/1739] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:40 2022] DELETE /identity/v3/projects/8707aae31fdc4f8d81446a45a3f80d34 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:40.906834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ad5b-5083-4503-a0f6-13c616ceb14f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:40.907121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ad5b-5083-4503-a0f6-13c616ceb14f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:40.907402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ad5b-5083-4503-a0f6-13c616ceb14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:41.016372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d3c2ad5b-5083-4503-a0f6-13c616ceb14f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:41.022168 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d3c2ad5b-5083-4503-a0f6-13c616ceb14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:41.047721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 868/1740] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Apr 29 18:43:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:41.048473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ffdad-dad9-49c5-9e42-3a389aa3f0da None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:41.065595 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-888ffdad-dad9-49c5-9e42-3a389aa3f0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:41.284913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 873/1741] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:41.289595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2ad5b-5083-4503-a0f6-13c616ceb14f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:41.298615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d3c2ad5b-5083-4503-a0f6-13c616ceb14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:41.343799 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2ad5b-5083-4503-a0f6-13c616ceb14f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:41.347482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ffdad-dad9-49c5-9e42-3a389aa3f0da None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:41.360014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fd951-e970-418a-b493-0b444075f8ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:41.362067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fd951-e970-418a-b493-0b444075f8ae service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:41.363627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fd951-e970-418a-b493-0b444075f8ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:41.364124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:41.365436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:41.365671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fd951-e970-418a-b493-0b444075f8ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:41.366741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:41.367544 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] PATH_INFO: `/v3/users/40157b0a618641828c5e45d40e823173` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:41.372586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40157b0a618641828c5e45d40e823173)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:41.374838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:41.381804 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418fd951-e970-418a-b493-0b444075f8ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:41.387075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418fd951-e970-418a-b493-0b444075f8ae service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:41.413599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 874/1742] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:41.415078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fd951-e970-418a-b493-0b444075f8ae service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:41.466664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fd951-e970-418a-b493-0b444075f8ae service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:41.488191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d9beb-2d53-4c7d-b4c0-a9a8944942b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:41.490580 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d9beb-2d53-4c7d-b4c0-a9a8944942b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:41.492100 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d9beb-2d53-4c7d-b4c0-a9a8944942b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:41.492548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d9beb-2d53-4c7d-b4c0-a9a8944942b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:41.493794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d9beb-2d53-4c7d-b4c0-a9a8944942b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:41.496343 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d9beb-2d53-4c7d-b4c0-a9a8944942b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:41.507475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d9beb-2d53-4c7d-b4c0-a9a8944942b0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:41.527710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40157b0a618641828c5e45d40e823173'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:41.529062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40157b0a618641828c5e45d40e823173'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:41.551173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 875/1743] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:41.564438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e9a70-4843-4b70-97b0-acc92bd8114d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:41.564933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e9a70-4843-4b70-97b0-acc92bd8114d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:41.566914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e9a70-4843-4b70-97b0-acc92bd8114d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:41.577641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40157b0a618641828c5e45d40e823173'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:41.604003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40157b0a618641828c5e45d40e823173'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:41.615230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ee2e9a70-4843-4b70-97b0-acc92bd8114d None None] MFA Rules not processed for user `64c8eec4efab42bfa5d9e9b157883dd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:41.616883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ee2e9a70-4843-4b70-97b0-acc92bd8114d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:41.627496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40157b0a618641828c5e45d40e823173'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:41.637815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 869/1744] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:41 2022] DELETE /identity/v3/users/40157b0a618641828c5e45d40e823173 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:41.638973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:41.643530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:41.763236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 876/1745] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:41.800631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635cfb1-6e7d-41d3-9c53-b58c4a6faab4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:41.810601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4f26f-a6f8-4d96-a9b4-a42e1e307f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:41.811548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4f26f-a6f8-4d96-a9b4-a42e1e307f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:41.811977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4f26f-a6f8-4d96-a9b4-a42e1e307f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:41.812390 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4f26f-a6f8-4d96-a9b4-a42e1e307f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:41.813345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4f26f-a6f8-4d96-a9b4-a42e1e307f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:41.817914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4f26f-a6f8-4d96-a9b4-a42e1e307f30 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:41.836855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097b976-de5d-4c51-a5c4-d9927899290e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:41.837397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097b976-de5d-4c51-a5c4-d9927899290e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:41.837813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097b976-de5d-4c51-a5c4-d9927899290e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:41.880118 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a097b976-de5d-4c51-a5c4-d9927899290e None None] MFA Rules not processed for user `e9a2443e71c54075823f75fdb8df469e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:41.888017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a097b976-de5d-4c51-a5c4-d9927899290e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:41.908578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 870/1746] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:41 2022] POST /identity/v3/projects => generated 381 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:41.911473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4f26f-a6f8-4d96-a9b4-a42e1e307f30 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:42.149879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 877/1747] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:43:42.176087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4f26f-a6f8-4d96-a9b4-a42e1e307f30 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:42.194004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463d788-2c7e-4289-82dc-a1b75f343835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:42.194868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463d788-2c7e-4289-82dc-a1b75f343835 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:42.195157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463d788-2c7e-4289-82dc-a1b75f343835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:42.195445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463d788-2c7e-4289-82dc-a1b75f343835 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:42.196152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463d788-2c7e-4289-82dc-a1b75f343835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:42.207948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463d788-2c7e-4289-82dc-a1b75f343835 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:42.277132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097b976-de5d-4c51-a5c4-d9927899290e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:42.291974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097b976-de5d-4c51-a5c4-d9927899290e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:42.292682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 871/1748] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:41 2022] POST /identity/v3/users => generated 291 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:42.307734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddead07-4002-4532-ac80-4e97c01b1a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:42.311384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddead07-4002-4532-ac80-4e97c01b1a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:42.311862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddead07-4002-4532-ac80-4e97c01b1a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:42.312865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddead07-4002-4532-ac80-4e97c01b1a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:42.313787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddead07-4002-4532-ac80-4e97c01b1a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:42.319827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddead07-4002-4532-ac80-4e97c01b1a56 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:42.366093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463d788-2c7e-4289-82dc-a1b75f343835 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:42.366961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 878/1749] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:42.381840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463d788-2c7e-4289-82dc-a1b75f343835 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:42.397361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ed07f-e261-4c89-898f-99b786a2dd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:42.398494 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ed07f-e261-4c89-898f-99b786a2dd7a admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:42.398906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ed07f-e261-4c89-898f-99b786a2dd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:42.399346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ed07f-e261-4c89-898f-99b786a2dd7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:42.411636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ed07f-e261-4c89-898f-99b786a2dd7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:42.414464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ed07f-e261-4c89-898f-99b786a2dd7a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:42.461691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 872/1750] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:42.465671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddead07-4002-4532-ac80-4e97c01b1a56 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:42.476703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddead07-4002-4532-ac80-4e97c01b1a56 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:42.504201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d746e49-fc34-40db-bd66-435d7dd95403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:42.510809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d746e49-fc34-40db-bd66-435d7dd95403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:42.511924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d746e49-fc34-40db-bd66-435d7dd95403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:42.512221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d746e49-fc34-40db-bd66-435d7dd95403 admin admin] PATH_INFO: `/v3/projects/eea71ec2e1244e8c9c71fc658c3a67c5/users/016f322f53bf4483b4ba2db4594ece2d/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:42.591566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d746e49-fc34-40db-bd66-435d7dd95403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea71ec2e1244e8c9c71fc658c3a67c5, user_id=016f322f53bf4483b4ba2db4594ece2d, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:42.614746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d746e49-fc34-40db-bd66-435d7dd95403 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:42.683456 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 879/1751] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:42 2022] PUT /identity/v3/projects/eea71ec2e1244e8c9c71fc658c3a67c5/users/016f322f53bf4483b4ba2db4594ece2d/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:42.694304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf33c77-5e62-447f-a391-672fd0afb6fc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:42.694652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf33c77-5e62-447f-a391-672fd0afb6fc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:42.694859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf33c77-5e62-447f-a391-672fd0afb6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:42.789061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-caf33c77-5e62-447f-a391-672fd0afb6fc None None] MFA Rules not processed for user `016f322f53bf4483b4ba2db4594ece2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:42.796376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-caf33c77-5e62-447f-a391-672fd0afb6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:43.111447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d746e49-fc34-40db-bd66-435d7dd95403 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:43.113461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3d746e49-fc34-40db-bd66-435d7dd95403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:43.143524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 873/1752] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:43.152603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf33c77-5e62-447f-a391-672fd0afb6fc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:43.370567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d746e49-fc34-40db-bd66-435d7dd95403 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:43.397320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:43.398152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:43.398442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:43.398722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] PATH_INFO: `/v3/projects/3b2bec523ca847208dd2402fce26185e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:43.406887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b2bec523ca847208dd2402fce26185e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:43.415977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:43.417516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf33c77-5e62-447f-a391-672fd0afb6fc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:43.437706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba18c77-5503-41c2-882a-1e879282ceb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:43.438373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba18c77-5503-41c2-882a-1e879282ceb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:43.438655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba18c77-5503-41c2-882a-1e879282ceb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:43.438924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba18c77-5503-41c2-882a-1e879282ceb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:43.439627 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba18c77-5503-41c2-882a-1e879282ceb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:43.441580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba18c77-5503-41c2-882a-1e879282ceb9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:43.547214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 874/1753] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:43 2022] POST /identity/v3/projects => generated 377 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:43.554388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba18c77-5503-41c2-882a-1e879282ceb9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:43.630824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b2bec523ca847208dd2402fce26185e', 'request_id': 'req-4bd5a816-8d7b-493f-ae35-d23f474af5cf'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:43.652840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b2bec523ca847208dd2402fce26185e', 'request_id': 'req-4bd5a816-8d7b-493f-ae35-d23f474af5cf'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:43.700070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b2bec523ca847208dd2402fce26185e', 'request_id': 'req-4bd5a816-8d7b-493f-ae35-d23f474af5cf'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:43.756932 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba18c77-5503-41c2-882a-1e879282ceb9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:43.762717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] The token cache is being invalidate because project 3b2bec523ca847208dd2402fce26185e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:43.763470 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:43.768269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4bd5a816-8d7b-493f-ae35-d23f474af5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:43.781756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 880/1754] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:43 2022] DELETE /identity/v3/projects/3b2bec523ca847208dd2402fce26185e => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:43:43.786731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d17d-320f-4780-8082-ec7549e40cad None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:43.787027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d17d-320f-4780-8082-ec7549e40cad None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:43.787308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d17d-320f-4780-8082-ec7549e40cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:43.891333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bb04d17d-320f-4780-8082-ec7549e40cad None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:43.895932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bb04d17d-320f-4780-8082-ec7549e40cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:43.899587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c4070-4c63-40b9-910b-19504a6637b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:43.902113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c4070-4c63-40b9-910b-19504a6637b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:43.903205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c4070-4c63-40b9-910b-19504a6637b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:43.903624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c4070-4c63-40b9-910b-19504a6637b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:43.904900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c4070-4c63-40b9-910b-19504a6637b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:43.909486 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c4070-4c63-40b9-910b-19504a6637b0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:43.942089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 881/1755] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:43.948963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13fba-bc69-43e9-86f6-803658edf81e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:43.949672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13fba-bc69-43e9-86f6-803658edf81e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:43.951645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13fba-bc69-43e9-86f6-803658edf81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:43.967173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 875/1756] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:43 2022] POST /identity/v3/users => generated 295 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:43.971709 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c4070-4c63-40b9-910b-19504a6637b0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:43.979639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-719c4070-4c63-40b9-910b-19504a6637b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:44.023318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c4070-4c63-40b9-910b-19504a6637b0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:44.031585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d0a13fba-bc69-43e9-86f6-803658edf81e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:44.036962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d0a13fba-bc69-43e9-86f6-803658edf81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:44.061147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884c66a-3133-4099-a4f9-8831ce5844e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:44.071212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 882/1757] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:44.073895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a13fba-bc69-43e9-86f6-803658edf81e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:44.074291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884c66a-3133-4099-a4f9-8831ce5844e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:44.074996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884c66a-3133-4099-a4f9-8831ce5844e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:44.077556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884c66a-3133-4099-a4f9-8831ce5844e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:44.082013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884c66a-3133-4099-a4f9-8831ce5844e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:44.089537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884c66a-3133-4099-a4f9-8831ce5844e9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:44.112704 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a13fba-bc69-43e9-86f6-803658edf81e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:44.127836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037b990-66b6-44f4-8967-b5a864247a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:44.127836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037b990-66b6-44f4-8967-b5a864247a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:44.127836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037b990-66b6-44f4-8967-b5a864247a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:44.128201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037b990-66b6-44f4-8967-b5a864247a30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:44.129462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037b990-66b6-44f4-8967-b5a864247a30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:44.129871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037b990-66b6-44f4-8967-b5a864247a30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:44.130291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 876/1758] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:44.133064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7b1ca-8194-4d3e-aebb-43626271d555 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:44.135223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7b1ca-8194-4d3e-aebb-43626271d555 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:44.135652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7b1ca-8194-4d3e-aebb-43626271d555 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:44.138895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c037b990-66b6-44f4-8967-b5a864247a30 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:44.141472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 877/1759] 10.222.0.50 () {54 vars in 912 bytes} [Fri Apr 29 18:43:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:43:44.142564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7b1ca-8194-4d3e-aebb-43626271d555 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:44.171995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 883/1760] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:44.172961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7b1ca-8194-4d3e-aebb-43626271d555 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:44.173656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bbc3f-eab2-463e-bbee-3ec9f343ce6f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:44.173966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bbc3f-eab2-463e-bbee-3ec9f343ce6f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:44.174272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bbc3f-eab2-463e-bbee-3ec9f343ce6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:44.191130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4863c-ff9c-4231-a869-ccb7f57b5c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:44.192427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4863c-ff9c-4231-a869-ccb7f57b5c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:44.207261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4863c-ff9c-4231-a869-ccb7f57b5c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:44.207756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4863c-ff9c-4231-a869-ccb7f57b5c37 admin admin] PATH_INFO: `/v3/projects/ad4227915d4f4066b089b2d9e85d5487/users/6b914d4825fa41eb9c29e61bd2c268d5/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:44.233954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cc3bbc3f-eab2-463e-bbee-3ec9f343ce6f None None] MFA Rules not processed for user `07849b2925334763b119111ed52dd34c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:44.237809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cc3bbc3f-eab2-463e-bbee-3ec9f343ce6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:44.280707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4863c-ff9c-4231-a869-ccb7f57b5c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad4227915d4f4066b089b2d9e85d5487, user_id=6b914d4825fa41eb9c29e61bd2c268d5, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:44.289884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4863c-ff9c-4231-a869-ccb7f57b5c37 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:44.338786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 878/1761] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:44 2022] PUT /identity/v3/projects/ad4227915d4f4066b089b2d9e85d5487/users/6b914d4825fa41eb9c29e61bd2c268d5/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:44.340635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4863c-ff9c-4231-a869-ccb7f57b5c37 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:44.386807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 884/1762] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:43:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:44.387714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bbc3f-eab2-463e-bbee-3ec9f343ce6f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:44.420000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bbc3f-eab2-463e-bbee-3ec9f343ce6f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:44.509663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4863c-ff9c-4231-a869-ccb7f57b5c37 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:44.520061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7dbc4-89e8-4670-83be-43c97e44df2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:44.520776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7dbc4-89e8-4670-83be-43c97e44df2c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:44.521055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7dbc4-89e8-4670-83be-43c97e44df2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:44.521329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7dbc4-89e8-4670-83be-43c97e44df2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:44.522027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7dbc4-89e8-4670-83be-43c97e44df2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:44.523941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7dbc4-89e8-4670-83be-43c97e44df2c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:44.582676 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 879/1763] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:44 2022] POST /identity/v3/projects => generated 379 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:44.587204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7dbc4-89e8-4670-83be-43c97e44df2c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:44.595619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e7e7dbc4-89e8-4670-83be-43c97e44df2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:44.739574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7dbc4-89e8-4670-83be-43c97e44df2c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:44.742287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ad901-def5-49c5-8c47-3f547ca7d564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:44.743246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad901-def5-49c5-8c47-3f547ca7d564 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:44.743772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad901-def5-49c5-8c47-3f547ca7d564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:44.744240 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad901-def5-49c5-8c47-3f547ca7d564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:44.745310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ad901-def5-49c5-8c47-3f547ca7d564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:44.748024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ad901-def5-49c5-8c47-3f547ca7d564 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:44.748312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17807fa-f949-4fb4-851d-3718307531c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:44.752200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17807fa-f949-4fb4-851d-3718307531c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:44.752200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17807fa-f949-4fb4-851d-3718307531c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:44.752200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17807fa-f949-4fb4-851d-3718307531c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:44.765371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17807fa-f949-4fb4-851d-3718307531c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:44.767041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17807fa-f949-4fb4-851d-3718307531c1 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:44.776061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 885/1764] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:44.777253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ad901-def5-49c5-8c47-3f547ca7d564 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:44.803725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 880/1765] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:44.804215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17807fa-f949-4fb4-851d-3718307531c1 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:44.808030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ad901-def5-49c5-8c47-3f547ca7d564 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:44.958363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17807fa-f949-4fb4-851d-3718307531c1 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:44.991340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bceede4-bdb8-481c-9575-a7d730482568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:44.991586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca20759-db52-4852-8e07-4da65c911614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:44.992145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca20759-db52-4852-8e07-4da65c911614 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:44.992414 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca20759-db52-4852-8e07-4da65c911614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:44.992741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bceede4-bdb8-481c-9575-a7d730482568 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:44.993115 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bceede4-bdb8-481c-9575-a7d730482568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:44.993478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bceede4-bdb8-481c-9575-a7d730482568 admin admin] PATH_INFO: `/v3/projects/ad4227915d4f4066b089b2d9e85d5487/users/6b914d4825fa41eb9c29e61bd2c268d5/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:44.996565 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca20759-db52-4852-8e07-4da65c911614 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:44.996697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bceede4-bdb8-481c-9575-a7d730482568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad4227915d4f4066b089b2d9e85d5487, user_id=6b914d4825fa41eb9c29e61bd2c268d5, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:44.996785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca20759-db52-4852-8e07-4da65c911614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:44.998728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca20759-db52-4852-8e07-4da65c911614 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:44.999475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bceede4-bdb8-481c-9575-a7d730482568 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:45.027265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 881/1766] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:44 2022] PUT /identity/v3/projects/ad4227915d4f4066b089b2d9e85d5487/users/6b914d4825fa41eb9c29e61bd2c268d5/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:45.040291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411c2ae-b7d0-4933-ae6b-89de66845093 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:45.040716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411c2ae-b7d0-4933-ae6b-89de66845093 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:45.041124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411c2ae-b7d0-4933-ae6b-89de66845093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:45.090330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 886/1767] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:44 2022] POST /identity/v3/users => generated 290 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:45.096320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca20759-db52-4852-8e07-4da65c911614 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:45.115232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca20759-db52-4852-8e07-4da65c911614 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:45.120404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c411c2ae-b7d0-4933-ae6b-89de66845093 None None] MFA Rules not processed for user `6b914d4825fa41eb9c29e61bd2c268d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:45.122168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c411c2ae-b7d0-4933-ae6b-89de66845093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:45.236452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb4d20-1a8b-4c2b-8ccf-dce250757bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:45.238041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb4d20-1a8b-4c2b-8ccf-dce250757bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:45.240017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb4d20-1a8b-4c2b-8ccf-dce250757bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:45.240017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb4d20-1a8b-4c2b-8ccf-dce250757bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:45.244525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb4d20-1a8b-4c2b-8ccf-dce250757bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:45.251520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb4d20-1a8b-4c2b-8ccf-dce250757bb4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:45.292438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 887/1768] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:45.297068 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb4d20-1a8b-4c2b-8ccf-dce250757bb4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:45.324517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb4d20-1a8b-4c2b-8ccf-dce250757bb4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:45.328779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 882/1769] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:43:45.343198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411c2ae-b7d0-4933-ae6b-89de66845093 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:45.344712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9601c0f-d2e0-4c40-9fc3-52e58c3a3321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:45.345433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9601c0f-d2e0-4c40-9fc3-52e58c3a3321 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:45.345680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c411c2ae-b7d0-4933-ae6b-89de66845093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:45.345800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9601c0f-d2e0-4c40-9fc3-52e58c3a3321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:45.346047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9601c0f-d2e0-4c40-9fc3-52e58c3a3321 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:45.352995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9601c0f-d2e0-4c40-9fc3-52e58c3a3321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:45.355279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9601c0f-d2e0-4c40-9fc3-52e58c3a3321 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:45.371990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 888/1770] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:45.379496 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9601c0f-d2e0-4c40-9fc3-52e58c3a3321 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:45.406110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9601c0f-d2e0-4c40-9fc3-52e58c3a3321 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:45.429800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a23a8-5787-41e4-9d86-0e7ad5ca66e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:45.431444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a23a8-5787-41e4-9d86-0e7ad5ca66e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:45.432296 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a23a8-5787-41e4-9d86-0e7ad5ca66e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:45.432625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a23a8-5787-41e4-9d86-0e7ad5ca66e5 admin admin] PATH_INFO: `/v3/projects/61de61544d2140e88a73e96e06e3a9b9/users/bf050127ee6049e6bf12b303f4972532/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:45.511004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a23a8-5787-41e4-9d86-0e7ad5ca66e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61de61544d2140e88a73e96e06e3a9b9, user_id=bf050127ee6049e6bf12b303f4972532, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:45.528382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a23a8-5787-41e4-9d86-0e7ad5ca66e5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:45.570672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411c2ae-b7d0-4933-ae6b-89de66845093 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:45.586496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a69ae-e671-4990-8d02-286ac5d37be6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:45.592091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a69ae-e671-4990-8d02-286ac5d37be6 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:45.596524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 889/1771] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:45 2022] PUT /identity/v3/projects/61de61544d2140e88a73e96e06e3a9b9/users/bf050127ee6049e6bf12b303f4972532/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:45.610641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a69ae-e671-4990-8d02-286ac5d37be6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:45.612297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77718794-2873-46d8-9fba-b1bb806d074a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:45.612621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77718794-2873-46d8-9fba-b1bb806d074a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:45.614094 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77718794-2873-46d8-9fba-b1bb806d074a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:45.616728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a69ae-e671-4990-8d02-286ac5d37be6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:45.662795 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-77718794-2873-46d8-9fba-b1bb806d074a None None] MFA Rules not processed for user `bf050127ee6049e6bf12b303f4972532`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:45.664440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-77718794-2873-46d8-9fba-b1bb806d074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:45.670069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a69ae-e671-4990-8d02-286ac5d37be6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:45.671706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a69ae-e671-4990-8d02-286ac5d37be6 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:45.686428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 883/1772] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:45.812595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 890/1773] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:45.818953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a69ae-e671-4990-8d02-286ac5d37be6 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:45.845672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a69ae-e671-4990-8d02-286ac5d37be6 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:46.054778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f7b11-49af-4ca4-8f02-099ca8a60c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:46.055470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f7b11-49af-4ca4-8f02-099ca8a60c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:46.055755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f7b11-49af-4ca4-8f02-099ca8a60c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:46.056030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f7b11-49af-4ca4-8f02-099ca8a60c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:46.056778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f7b11-49af-4ca4-8f02-099ca8a60c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:46.058721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f7b11-49af-4ca4-8f02-099ca8a60c61 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:46.121938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 884/1774] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:45 2022] POST /identity/v3/projects => generated 379 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:46.123233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77718794-2873-46d8-9fba-b1bb806d074a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:46.138327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77718794-2873-46d8-9fba-b1bb806d074a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:46.347777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe6c97-be24-4ec6-9414-65c591b480d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:46.350448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe6c97-be24-4ec6-9414-65c591b480d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:46.350846 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe6c97-be24-4ec6-9414-65c591b480d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:46.351228 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe6c97-be24-4ec6-9414-65c591b480d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:46.353420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe6c97-be24-4ec6-9414-65c591b480d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:46.358174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe6c97-be24-4ec6-9414-65c591b480d7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:46.413959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 891/1775] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:46 2022] POST /identity/v3/users => generated 296 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:46.419732 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f7b11-49af-4ca4-8f02-099ca8a60c61 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:46.430342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe6c97-be24-4ec6-9414-65c591b480d7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:46.459677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f7b11-49af-4ca4-8f02-099ca8a60c61 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:46.480375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48293a-9f9a-448b-8a87-ed8403d7da5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:46.481279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48293a-9f9a-448b-8a87-ed8403d7da5c admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:46.484892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48293a-9f9a-448b-8a87-ed8403d7da5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:46.485178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48293a-9f9a-448b-8a87-ed8403d7da5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:46.488382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48293a-9f9a-448b-8a87-ed8403d7da5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:46.490111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48293a-9f9a-448b-8a87-ed8403d7da5c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:46.554898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 885/1776] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:46.556071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48293a-9f9a-448b-8a87-ed8403d7da5c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:46.580819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48293a-9f9a-448b-8a87-ed8403d7da5c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:46.611697 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe6c97-be24-4ec6-9414-65c591b480d7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:46.612184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177b03-30bc-4e66-99df-8049dc2a31cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:46.621542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177b03-30bc-4e66-99df-8049dc2a31cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:46.625273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177b03-30bc-4e66-99df-8049dc2a31cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:46.626500 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177b03-30bc-4e66-99df-8049dc2a31cb admin admin] PATH_INFO: `/v3/projects/06eb6a8f9f9542ff856902d0705409e8/users/90c15197eccc4ff4aa128ef4a0bee1bb/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:46.635345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624792ff-f7cd-45b3-ba0b-36c41eacbeeb tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b914d4825fa41eb9c29e61bd2c268d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4227915d4f4066b089b2d9e85d5487', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:46.638772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624792ff-f7cd-45b3-ba0b-36c41eacbeeb tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:46.642544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624792ff-f7cd-45b3-ba0b-36c41eacbeeb tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:46.642544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624792ff-f7cd-45b3-ba0b-36c41eacbeeb tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:46.642544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624792ff-f7cd-45b3-ba0b-36c41eacbeeb tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:46.642843 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624792ff-f7cd-45b3-ba0b-36c41eacbeeb tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:46.651609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624792ff-f7cd-45b3-ba0b-36c41eacbeeb tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:46.672314 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08177b03-30bc-4e66-99df-8049dc2a31cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=06eb6a8f9f9542ff856902d0705409e8, user_id=90c15197eccc4ff4aa128ef4a0bee1bb, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:46.680002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08177b03-30bc-4e66-99df-8049dc2a31cb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:46.713622 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 886/1777] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:46 2022] PUT /identity/v3/projects/06eb6a8f9f9542ff856902d0705409e8/users/90c15197eccc4ff4aa128ef4a0bee1bb/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:46.714139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 892/1778] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:43:46.718749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177b03-30bc-4e66-99df-8049dc2a31cb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:46.721243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624792ff-f7cd-45b3-ba0b-36c41eacbeeb tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:46.763659 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624792ff-f7cd-45b3-ba0b-36c41eacbeeb tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:46.883993 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177b03-30bc-4e66-99df-8049dc2a31cb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:46.885937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15682a1-f4d4-4de0-8963-3bd30e7e33ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:46.886752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15682a1-f4d4-4de0-8963-3bd30e7e33ab admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:46.887137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15682a1-f4d4-4de0-8963-3bd30e7e33ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:46.887517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15682a1-f4d4-4de0-8963-3bd30e7e33ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:46.888368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15682a1-f4d4-4de0-8963-3bd30e7e33ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:46.891494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15682a1-f4d4-4de0-8963-3bd30e7e33ab admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:46.893293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407e91e-31f0-471a-a50b-e9ec1d4836ed tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b914d4825fa41eb9c29e61bd2c268d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4227915d4f4066b089b2d9e85d5487', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:46.894311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407e91e-31f0-471a-a50b-e9ec1d4836ed tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:46.894959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407e91e-31f0-471a-a50b-e9ec1d4836ed tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:46.895380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407e91e-31f0-471a-a50b-e9ec1d4836ed tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:46.897078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407e91e-31f0-471a-a50b-e9ec1d4836ed tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:46.899909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407e91e-31f0-471a-a50b-e9ec1d4836ed tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:46.933554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 893/1779] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:46.943182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15682a1-f4d4-4de0-8963-3bd30e7e33ab admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:46.949722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 887/1780] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:46 2022] POST /identity/v3/projects => generated 355 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:46.955993 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407e91e-31f0-471a-a50b-e9ec1d4836ed tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:46.961498 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15682a1-f4d4-4de0-8963-3bd30e7e33ab admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:47.076388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407e91e-31f0-471a-a50b-e9ec1d4836ed tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:47.108040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73c90b-0cb5-4785-9df0-d7c8e663e6c9 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b914d4825fa41eb9c29e61bd2c268d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4227915d4f4066b089b2d9e85d5487', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:47.110497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73c90b-0cb5-4785-9df0-d7c8e663e6c9 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:47.111078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73c90b-0cb5-4785-9df0-d7c8e663e6c9 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:47.111502 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73c90b-0cb5-4785-9df0-d7c8e663e6c9 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:47.113025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73c90b-0cb5-4785-9df0-d7c8e663e6c9 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:47.113527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73c90b-0cb5-4785-9df0-d7c8e663e6c9 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:47.116243 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73c90b-0cb5-4785-9df0-d7c8e663e6c9 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:47.147610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 888/1781] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:47.165878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161272cb-2651-4cf3-9601-733b8bda9ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:47.167078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161272cb-2651-4cf3-9601-733b8bda9ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:47.176869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161272cb-2651-4cf3-9601-733b8bda9ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:47.177373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161272cb-2651-4cf3-9601-733b8bda9ba2 admin admin] PATH_INFO: `/v3/projects/06eb6a8f9f9542ff856902d0705409e8/users/90c15197eccc4ff4aa128ef4a0bee1bb/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:47.181182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161272cb-2651-4cf3-9601-733b8bda9ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06eb6a8f9f9542ff856902d0705409e8, user_id=90c15197eccc4ff4aa128ef4a0bee1bb, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:47.182855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161272cb-2651-4cf3-9601-733b8bda9ba2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:47.218221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 894/1782] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:46 2022] PUT /identity/v3/projects/06eb6a8f9f9542ff856902d0705409e8/users/90c15197eccc4ff4aa128ef4a0bee1bb/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:47.221561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9161f-3486-47d2-af78-da54406c1080 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:47.222565 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9161f-3486-47d2-af78-da54406c1080 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:47.222852 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9161f-3486-47d2-af78-da54406c1080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:47.269288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161272cb-2651-4cf3-9601-733b8bda9ba2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:47.338164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8ff9161f-3486-47d2-af78-da54406c1080 None None] MFA Rules not processed for user `90c15197eccc4ff4aa128ef4a0bee1bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:47.343783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8ff9161f-3486-47d2-af78-da54406c1080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:47.415825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161272cb-2651-4cf3-9601-733b8bda9ba2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:47.421042 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b914d4825fa41eb9c29e61bd2c268d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4227915d4f4066b089b2d9e85d5487', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:47.422092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:47.422607 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:47.423117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] PATH_INFO: `/v3/projects/9f9e502dfb4e455e9ccebef587fd5b06` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:47.426245 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9f9e502dfb4e455e9ccebef587fd5b06)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:47.431653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:47.543243 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 889/1783] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:47.557574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9161f-3486-47d2-af78-da54406c1080 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:47.558930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8ff9161f-3486-47d2-af78-da54406c1080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:47.574222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9e502dfb4e455e9ccebef587fd5b06', 'request_id': 'req-94123811-7f75-4ea3-8f12-0bfd79081da5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:47.599059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9e502dfb4e455e9ccebef587fd5b06', 'request_id': 'req-94123811-7f75-4ea3-8f12-0bfd79081da5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:47.618335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9e502dfb4e455e9ccebef587fd5b06', 'request_id': 'req-94123811-7f75-4ea3-8f12-0bfd79081da5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:47.654967 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] The token cache is being invalidate because project 9f9e502dfb4e455e9ccebef587fd5b06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:47.655728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:47.668726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94123811-7f75-4ea3-8f12-0bfd79081da5 tempest-VolumeQuotaClassesTest-54310287 tempest-VolumeQuotaClassesTest-54310287-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:47.670760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 895/1784] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:47 2022] DELETE /identity/v3/projects/9f9e502dfb4e455e9ccebef587fd5b06 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:47.696849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9161f-3486-47d2-af78-da54406c1080 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:47.799003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdc263-7408-4a42-8db6-a1f175bed5a4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:47.799424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdc263-7408-4a42-8db6-a1f175bed5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:47.799785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdc263-7408-4a42-8db6-a1f175bed5a4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:47.801369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 896/1785] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Apr 29 18:43:47.815343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bc5d7-6603-43c0-a70f-8c59dbd29177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:47.816018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bc5d7-6603-43c0-a70f-8c59dbd29177 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:47.817307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bc5d7-6603-43c0-a70f-8c59dbd29177 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:47.818570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bc5d7-6603-43c0-a70f-8c59dbd29177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:47.820848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a0dfc-3490-4224-a6d2-c0df6649383b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:47.821162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a0dfc-3490-4224-a6d2-c0df6649383b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:47.821437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a0dfc-3490-4224-a6d2-c0df6649383b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:47.824182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8f3bc5d7-6603-43c0-a70f-8c59dbd29177 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:47.872245 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c76a0dfc-3490-4224-a6d2-c0df6649383b None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:47.875454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c76a0dfc-3490-4224-a6d2-c0df6649383b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:47.931773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bc5d7-6603-43c0-a70f-8c59dbd29177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:47.934035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bc5d7-6603-43c0-a70f-8c59dbd29177 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:47.971319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 897/1786] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:43:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:47.981528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 890/1787] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:43:47.993307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a0dfc-3490-4224-a6d2-c0df6649383b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:47.996935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c76a0dfc-3490-4224-a6d2-c0df6649383b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:48.018579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a0dfc-3490-4224-a6d2-c0df6649383b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:48.034129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe26398-0979-4ad4-96fb-e38b5389bf6d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:48.034430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe26398-0979-4ad4-96fb-e38b5389bf6d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:48.034730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe26398-0979-4ad4-96fb-e38b5389bf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:48.044463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f2a90-2509-4979-ad63-fc443d9e06b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:48.045350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f2a90-2509-4979-ad63-fc443d9e06b3 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:48.045733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f2a90-2509-4979-ad63-fc443d9e06b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:48.046957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f2a90-2509-4979-ad63-fc443d9e06b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:48.064873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f2a90-2509-4979-ad63-fc443d9e06b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:48.066530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f2a90-2509-4979-ad63-fc443d9e06b3 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:48.070920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-2fe26398-0979-4ad4-96fb-e38b5389bf6d None None] MFA Rules not processed for user `49aaea67fba64d2583d2023505ce7779`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:48.072445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2fe26398-0979-4ad4-96fb-e38b5389bf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:48.085165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 898/1788] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:48.086614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f2a90-2509-4979-ad63-fc443d9e06b3 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:48.087928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8e9f2a90-2509-4979-ad63-fc443d9e06b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:48.176871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f2a90-2509-4979-ad63-fc443d9e06b3 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:48.182279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:48.183322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:48.183671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:48.184007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] PATH_INFO: `/v3/users/016f322f53bf4483b4ba2db4594ece2d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:48.185374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=016f322f53bf4483b4ba2db4594ece2d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:48.187532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:48.193343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 891/1789] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:48.194406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe26398-0979-4ad4-96fb-e38b5389bf6d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:48.195873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2fe26398-0979-4ad4-96fb-e38b5389bf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:48.209656 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe26398-0979-4ad4-96fb-e38b5389bf6d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:48.213470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa1668-36e0-444c-9f10-5f71327658b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:48.214225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa1668-36e0-444c-9f10-5f71327658b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:48.214570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa1668-36e0-444c-9f10-5f71327658b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:48.214911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa1668-36e0-444c-9f10-5f71327658b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:48.216962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6cfa1668-36e0-444c-9f10-5f71327658b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:48.264450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016f322f53bf4483b4ba2db4594ece2d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:48.265156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016f322f53bf4483b4ba2db4594ece2d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:48.280339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016f322f53bf4483b4ba2db4594ece2d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:48.303280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa1668-36e0-444c-9f10-5f71327658b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:48.305011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa1668-36e0-444c-9f10-5f71327658b2 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:48.315695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016f322f53bf4483b4ba2db4594ece2d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:48.337346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016f322f53bf4483b4ba2db4594ece2d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:48.339997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 892/1790] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:48.346878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 899/1791] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:48 2022] DELETE /identity/v3/users/016f322f53bf4483b4ba2db4594ece2d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:48.352602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12323c74-ac99-463f-bb31-b54672086be1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:48.353056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12323c74-ac99-463f-bb31-b54672086be1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:48.353882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12323c74-ac99-463f-bb31-b54672086be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:48.395620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-12323c74-ac99-463f-bb31-b54672086be1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:48.397061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-12323c74-ac99-463f-bb31-b54672086be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:48.643088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 893/1792] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:48.651668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:48.654588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:48.847198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515fe7a-5887-4895-9835-c4b2d993cc47 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:48.866145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ea4c3-5d05-4664-a09f-5c4f02e52186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:48.869117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea4c3-5d05-4664-a09f-5c4f02e52186 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:48.869669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea4c3-5d05-4664-a09f-5c4f02e52186 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:48.870914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea4c3-5d05-4664-a09f-5c4f02e52186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:48.907982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ea4c3-5d05-4664-a09f-5c4f02e52186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:48.911086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ea4c3-5d05-4664-a09f-5c4f02e52186 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:48.959796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 900/1793] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:49.768268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12323c74-ac99-463f-bb31-b54672086be1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:49.771613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-12323c74-ac99-463f-bb31-b54672086be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:49.822170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12323c74-ac99-463f-bb31-b54672086be1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:49.837830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:49.838736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:49.839009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:49.839285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] PATH_INFO: `/v3/projects/eea71ec2e1244e8c9c71fc658c3a67c5` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:49.846765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea71ec2e1244e8c9c71fc658c3a67c5)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:49.849975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:49.940671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea71ec2e1244e8c9c71fc658c3a67c5', 'request_id': 'req-1e2f1fc4-2479-4900-938a-416f95c28092'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:49.955943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea71ec2e1244e8c9c71fc658c3a67c5', 'request_id': 'req-1e2f1fc4-2479-4900-938a-416f95c28092'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:49.963204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea71ec2e1244e8c9c71fc658c3a67c5', 'request_id': 'req-1e2f1fc4-2479-4900-938a-416f95c28092'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:49.975696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] The token cache is being invalidate because project eea71ec2e1244e8c9c71fc658c3a67c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:49.976476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:49.988075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:49.990227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 894/1794] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:49 2022] DELETE /identity/v3/projects/eea71ec2e1244e8c9c71fc658c3a67c5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:49.994112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ea4c3-5d05-4664-a09f-5c4f02e52186 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:50.004806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ea4c3-5d05-4664-a09f-5c4f02e52186 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:50.094102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:50.095114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:50.095465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:50.095809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] PATH_INFO: `/v3/users/6b914d4825fa41eb9c29e61bd2c268d5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:50.097160 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b914d4825fa41eb9c29e61bd2c268d5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:50.099239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:50.178719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b914d4825fa41eb9c29e61bd2c268d5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:50.179857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b914d4825fa41eb9c29e61bd2c268d5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:50.199274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b914d4825fa41eb9c29e61bd2c268d5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:50.217614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b914d4825fa41eb9c29e61bd2c268d5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:50.239766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-402c7182-54c7-4c42-aa35-d537cc810aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b914d4825fa41eb9c29e61bd2c268d5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:50.246625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 901/1795] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:49 2022] DELETE /identity/v3/users/6b914d4825fa41eb9c29e61bd2c268d5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:43:50.674907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:50.676973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:50.695814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f1fc4-2479-4900-938a-416f95c28092 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:50.805348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:50.806417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:50.806941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:50.807273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] PATH_INFO: `/v3/projects/ad4227915d4f4066b089b2d9e85d5487` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:50.809424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad4227915d4f4066b089b2d9e85d5487)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:50.811653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:50.869911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad4227915d4f4066b089b2d9e85d5487', 'request_id': 'req-37085a66-09c0-48a8-9665-9aa3bb090062'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:50.880954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad4227915d4f4066b089b2d9e85d5487', 'request_id': 'req-37085a66-09c0-48a8-9665-9aa3bb090062'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:50.889953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad4227915d4f4066b089b2d9e85d5487', 'request_id': 'req-37085a66-09c0-48a8-9665-9aa3bb090062'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:50.904958 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] The token cache is being invalidate because project ad4227915d4f4066b089b2d9e85d5487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:50.905265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:50.906265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37085a66-09c0-48a8-9665-9aa3bb090062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:50.908433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 895/1796] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:50 2022] DELETE /identity/v3/projects/ad4227915d4f4066b089b2d9e85d5487 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:50.913439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac984f5-5fc7-4e3b-bd82-d88b67213154 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:50.913731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac984f5-5fc7-4e3b-bd82-d88b67213154 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:50.914178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac984f5-5fc7-4e3b-bd82-d88b67213154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:50.945714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5ac984f5-5fc7-4e3b-bd82-d88b67213154 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:50.949508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5ac984f5-5fc7-4e3b-bd82-d88b67213154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:51.094361 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 902/1797] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:51.102529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801e23-de03-47c2-a338-3ed77ca81d38 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:51.103068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801e23-de03-47c2-a338-3ed77ca81d38 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:51.103395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801e23-de03-47c2-a338-3ed77ca81d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:51.123333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f3801e23-de03-47c2-a338-3ed77ca81d38 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:51.127021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f3801e23-de03-47c2-a338-3ed77ca81d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:51.153033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 896/1798] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:51.154367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac984f5-5fc7-4e3b-bd82-d88b67213154 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:51.166232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac984f5-5fc7-4e3b-bd82-d88b67213154 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:51.175110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c1bb8-a205-432e-a0e5-16c63a7d317d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:51.176199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1bb8-a205-432e-a0e5-16c63a7d317d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:51.176485 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1bb8-a205-432e-a0e5-16c63a7d317d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:51.176785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1bb8-a205-432e-a0e5-16c63a7d317d admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:51.177711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1bb8-a205-432e-a0e5-16c63a7d317d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:51.178070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1bb8-a205-432e-a0e5-16c63a7d317d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:51.179931 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1bb8-a205-432e-a0e5-16c63a7d317d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:51.195707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 903/1799] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:51.200313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3801e23-de03-47c2-a338-3ed77ca81d38 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:51.211930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3801e23-de03-47c2-a338-3ed77ca81d38 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:51.219553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f98a5-2ae6-4ee6-aa54-c26e3b93fca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:51.221004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f98a5-2ae6-4ee6-aa54-c26e3b93fca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:51.221385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f98a5-2ae6-4ee6-aa54-c26e3b93fca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:51.221666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f98a5-2ae6-4ee6-aa54-c26e3b93fca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:51.222444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f98a5-2ae6-4ee6-aa54-c26e3b93fca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:51.224443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f98a5-2ae6-4ee6-aa54-c26e3b93fca3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:51.264634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 897/1800] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:51 2022] POST /identity/v3/projects => generated 401 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:51.267238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c1bb8-a205-432e-a0e5-16c63a7d317d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:51.278914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c1bb8-a205-432e-a0e5-16c63a7d317d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:51.339202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefd8d6-d69f-42e6-bcda-89ff346915c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:51.339897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefd8d6-d69f-42e6-bcda-89ff346915c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:51.340231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefd8d6-d69f-42e6-bcda-89ff346915c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:51.340530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caefd8d6-d69f-42e6-bcda-89ff346915c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:51.341230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefd8d6-d69f-42e6-bcda-89ff346915c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:51.343235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caefd8d6-d69f-42e6-bcda-89ff346915c2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:51.366344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 904/1801] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:51 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:51.370614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f98a5-2ae6-4ee6-aa54-c26e3b93fca3 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:51.381026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f98a5-2ae6-4ee6-aa54-c26e3b93fca3 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:51.385772 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dd456-6209-4e23-8262-b2da402544a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:51.386449 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dd456-6209-4e23-8262-b2da402544a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:51.386725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dd456-6209-4e23-8262-b2da402544a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:51.386991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dd456-6209-4e23-8262-b2da402544a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:51.387712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dd456-6209-4e23-8262-b2da402544a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:51.389475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dd456-6209-4e23-8262-b2da402544a3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:51.400375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 898/1802] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:51.404514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefd8d6-d69f-42e6-bcda-89ff346915c2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:51.416063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caefd8d6-d69f-42e6-bcda-89ff346915c2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:51.421038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd881311-e893-4f5a-bf4f-993ca10875d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:51.421723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd881311-e893-4f5a-bf4f-993ca10875d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:51.422001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd881311-e893-4f5a-bf4f-993ca10875d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:51.422271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd881311-e893-4f5a-bf4f-993ca10875d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:51.422988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd881311-e893-4f5a-bf4f-993ca10875d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:51.424623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd881311-e893-4f5a-bf4f-993ca10875d7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:51.439359 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 905/1803] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:51.446783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dd456-6209-4e23-8262-b2da402544a3 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:51.463227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dd456-6209-4e23-8262-b2da402544a3 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:51.470192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5db10-71e6-4613-9720-71d57c478856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:51.473285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5db10-71e6-4613-9720-71d57c478856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:51.473625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5db10-71e6-4613-9720-71d57c478856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:51.473907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5db10-71e6-4613-9720-71d57c478856 admin admin] PATH_INFO: `/v3/projects/7175d930c72449a3a9ab36f6d9c491d1/users/fffed55f2973495687fce09a195c76b5/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:51.490416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5db10-71e6-4613-9720-71d57c478856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7175d930c72449a3a9ab36f6d9c491d1, user_id=fffed55f2973495687fce09a195c76b5, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:51.493469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5db10-71e6-4613-9720-71d57c478856 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:51.510985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 899/1804] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:51 2022] PUT /identity/v3/projects/7175d930c72449a3a9ab36f6d9c491d1/users/fffed55f2973495687fce09a195c76b5/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:51.521707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68093224-aebc-43ee-9855-ee1b1d9451d0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:51.522001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68093224-aebc-43ee-9855-ee1b1d9451d0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:51.522262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68093224-aebc-43ee-9855-ee1b1d9451d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:51.588365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-68093224-aebc-43ee-9855-ee1b1d9451d0 None None] MFA Rules not processed for user `fffed55f2973495687fce09a195c76b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:51.589860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-68093224-aebc-43ee-9855-ee1b1d9451d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:51.717774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 906/1805] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:51.721854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5db10-71e6-4613-9720-71d57c478856 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:51.816381 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5db10-71e6-4613-9720-71d57c478856 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:51.819092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbd702-f9fb-4bf0-a0a9-d7a5330ed2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:51.819708 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbd702-f9fb-4bf0-a0a9-d7a5330ed2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:51.819952 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbd702-f9fb-4bf0-a0a9-d7a5330ed2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:51.820255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbd702-f9fb-4bf0-a0a9-d7a5330ed2d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:51.821184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbd702-f9fb-4bf0-a0a9-d7a5330ed2d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:51.823290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbd702-f9fb-4bf0-a0a9-d7a5330ed2d5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:51.854745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 900/1806] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:51 2022] POST /identity/v3/projects => generated 401 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:51.858688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68093224-aebc-43ee-9855-ee1b1d9451d0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:51.884937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68093224-aebc-43ee-9855-ee1b1d9451d0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:51.978625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cd06e-5c95-4b41-babf-7b55f9a91b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:51.979316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cd06e-5c95-4b41-babf-7b55f9a91b8d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:51.979597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cd06e-5c95-4b41-babf-7b55f9a91b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:51.980176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cd06e-5c95-4b41-babf-7b55f9a91b8d admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:51.980909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cd06e-5c95-4b41-babf-7b55f9a91b8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:51.982948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cd06e-5c95-4b41-babf-7b55f9a91b8d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:52.012111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 907/1807] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:51 2022] POST /identity/v3/users => generated 307 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:52.014294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbd702-f9fb-4bf0-a0a9-d7a5330ed2d5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:52.028146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbd702-f9fb-4bf0-a0a9-d7a5330ed2d5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:52.037246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380dab3-b525-4758-b23f-1f93a34d10fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:52.038461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380dab3-b525-4758-b23f-1f93a34d10fb admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:52.039074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380dab3-b525-4758-b23f-1f93a34d10fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:52.039653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380dab3-b525-4758-b23f-1f93a34d10fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:52.042344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380dab3-b525-4758-b23f-1f93a34d10fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:52.045540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380dab3-b525-4758-b23f-1f93a34d10fb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:52.062082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 901/1808] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:52.066723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cd06e-5c95-4b41-babf-7b55f9a91b8d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:52.079039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cd06e-5c95-4b41-babf-7b55f9a91b8d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:52.086729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc55685-2591-4ee3-9363-6303bf0650c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:52.089359 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55685-2591-4ee3-9363-6303bf0650c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:52.089359 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55685-2591-4ee3-9363-6303bf0650c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:52.089359 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55685-2591-4ee3-9363-6303bf0650c3 admin admin] PATH_INFO: `/v3/projects/57177c2dacd243508f3c61af94b64496/users/9bd61a9db1cb44b58e63cfffb1d1e877/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:52.109248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc55685-2591-4ee3-9363-6303bf0650c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57177c2dacd243508f3c61af94b64496, user_id=9bd61a9db1cb44b58e63cfffb1d1e877, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:52.112886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc55685-2591-4ee3-9363-6303bf0650c3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:52.137676 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 908/1809] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:52 2022] PUT /identity/v3/projects/57177c2dacd243508f3c61af94b64496/users/9bd61a9db1cb44b58e63cfffb1d1e877/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:43:52.139479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380dab3-b525-4758-b23f-1f93a34d10fb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:52.152791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380dab3-b525-4758-b23f-1f93a34d10fb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:52.243166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244211b-5cae-495c-949f-47a6436e3ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:52.243980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244211b-5cae-495c-949f-47a6436e3ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:52.244299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244211b-5cae-495c-949f-47a6436e3ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:52.244657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244211b-5cae-495c-949f-47a6436e3ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:52.245389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244211b-5cae-495c-949f-47a6436e3ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:52.247040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244211b-5cae-495c-949f-47a6436e3ab4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:52.280293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc55685-2591-4ee3-9363-6303bf0650c3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:52.290425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 902/1810] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:52.291775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc55685-2591-4ee3-9363-6303bf0650c3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:52.297504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbdfe6-b315-40db-9acc-607158fcbf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:52.298464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbdfe6-b315-40db-9acc-607158fcbf1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:52.298770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbdfe6-b315-40db-9acc-607158fcbf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:52.299043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbdfe6-b315-40db-9acc-607158fcbf1d admin admin] PATH_INFO: `/v3/projects/57177c2dacd243508f3c61af94b64496/users/9bd61a9db1cb44b58e63cfffb1d1e877/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:52.301169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbdfe6-b315-40db-9acc-607158fcbf1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=57177c2dacd243508f3c61af94b64496, user_id=9bd61a9db1cb44b58e63cfffb1d1e877, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:52.304048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbdfe6-b315-40db-9acc-607158fcbf1d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:52.315932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 909/1811] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:52 2022] PUT /identity/v3/projects/57177c2dacd243508f3c61af94b64496/users/9bd61a9db1cb44b58e63cfffb1d1e877/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:52.336905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3727a-85eb-4337-b81c-92305a93cfa8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:52.337313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3727a-85eb-4337-b81c-92305a93cfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:52.337666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3727a-85eb-4337-b81c-92305a93cfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:52.377339 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-94c3727a-85eb-4337-b81c-92305a93cfa8 None None] MFA Rules not processed for user `9bd61a9db1cb44b58e63cfffb1d1e877`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:52.379317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-94c3727a-85eb-4337-b81c-92305a93cfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:52.454867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 903/1812] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:52 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:52.465947 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbdfe6-b315-40db-9acc-607158fcbf1d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:52.467717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-40dbdfe6-b315-40db-9acc-607158fcbf1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:52.525966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbdfe6-b315-40db-9acc-607158fcbf1d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:52.530269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8b533-9af2-4a7d-bb72-32ffc9b02892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:52.530927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8b533-9af2-4a7d-bb72-32ffc9b02892 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:52.531211 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8b533-9af2-4a7d-bb72-32ffc9b02892 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:52.531487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8b533-9af2-4a7d-bb72-32ffc9b02892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:52.541211 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8b533-9af2-4a7d-bb72-32ffc9b02892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:52.542649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8b533-9af2-4a7d-bb72-32ffc9b02892 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:52.552550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 910/1813] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:52 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:52.632904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3727a-85eb-4337-b81c-92305a93cfa8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:52.643184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3727a-85eb-4337-b81c-92305a93cfa8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:52.646985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbc9ef-a6f1-47c3-b877-251a1685be66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:52.647630 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc9ef-a6f1-47c3-b877-251a1685be66 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:52.647904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc9ef-a6f1-47c3-b877-251a1685be66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:52.648168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc9ef-a6f1-47c3-b877-251a1685be66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:52.707417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc9ef-a6f1-47c3-b877-251a1685be66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:52.708922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc9ef-a6f1-47c3-b877-251a1685be66 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:52.718331 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 904/1814] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:53.012973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8b533-9af2-4a7d-bb72-32ffc9b02892 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:53.014798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e9e8b533-9af2-4a7d-bb72-32ffc9b02892 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:53.087926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8b533-9af2-4a7d-bb72-32ffc9b02892 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:53.093224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:53.093554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:53.093897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:53.094088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] PATH_INFO: `/v3/users/7a26abe32aee4386a9ea9e850f6a83a9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:53.095331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a26abe32aee4386a9ea9e850f6a83a9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:53.097274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:53.145392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a26abe32aee4386a9ea9e850f6a83a9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.146189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a26abe32aee4386a9ea9e850f6a83a9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.158339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a26abe32aee4386a9ea9e850f6a83a9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.166802 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a26abe32aee4386a9ea9e850f6a83a9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.173929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a26abe32aee4386a9ea9e850f6a83a9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.179913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 911/1815] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:53 2022] DELETE /identity/v3/users/7a26abe32aee4386a9ea9e850f6a83a9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:53.184433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7928f0-8795-4ecf-89a8-95f0c6236ebe None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:53.184911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7928f0-8795-4ecf-89a8-95f0c6236ebe None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:53.185008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7928f0-8795-4ecf-89a8-95f0c6236ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:53.203850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4b7928f0-8795-4ecf-89a8-95f0c6236ebe None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:53.205085 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4b7928f0-8795-4ecf-89a8-95f0c6236ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:53.270022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 905/1816] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:53.281722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:53.283738 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:53.359727 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284400c-7d49-4f3c-b79e-de5a3299021b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:53.364690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3cc75-2061-4d46-9f5b-e5500bc0d1be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:53.365363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3cc75-2061-4d46-9f5b-e5500bc0d1be service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:53.365641 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3cc75-2061-4d46-9f5b-e5500bc0d1be service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:53.365916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3cc75-2061-4d46-9f5b-e5500bc0d1be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:53.376819 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3cc75-2061-4d46-9f5b-e5500bc0d1be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:53.378394 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3cc75-2061-4d46-9f5b-e5500bc0d1be service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:53.390277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 912/1817] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:53.694855 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7928f0-8795-4ecf-89a8-95f0c6236ebe None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:53.696454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4b7928f0-8795-4ecf-89a8-95f0c6236ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:53.715129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7928f0-8795-4ecf-89a8-95f0c6236ebe None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:53.721470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:53.722615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:53.723019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:53.723352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] PATH_INFO: `/v3/projects/3b3df1ea2ae24c59901801dd9dae6ae0` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:53.724708 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3df1ea2ae24c59901801dd9dae6ae0)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:53.726833 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:53.772880 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3df1ea2ae24c59901801dd9dae6ae0', 'request_id': 'req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.781882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3df1ea2ae24c59901801dd9dae6ae0', 'request_id': 'req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.788176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3df1ea2ae24c59901801dd9dae6ae0', 'request_id': 'req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.798721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] The token cache is being invalidate because project 3b3df1ea2ae24c59901801dd9dae6ae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:53.798999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.799642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.801256 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 906/1818] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:53 2022] DELETE /identity/v3/projects/3b3df1ea2ae24c59901801dd9dae6ae0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:53.805484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3cc75-2061-4d46-9f5b-e5500bc0d1be service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:53.815968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3cc75-2061-4d46-9f5b-e5500bc0d1be service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:53.884649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:53.885558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:53.885833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:53.886112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] PATH_INFO: `/v3/users/2d688d6d8b63472a8512f0811efae197` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:53.887350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d688d6d8b63472a8512f0811efae197)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:53.889491 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:53.940842 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d688d6d8b63472a8512f0811efae197'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.941594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d688d6d8b63472a8512f0811efae197'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.953579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d688d6d8b63472a8512f0811efae197'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.962184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d688d6d8b63472a8512f0811efae197'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.968957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-19075f6a-94d7-4ef4-93f9-3f36e2f9236e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d688d6d8b63472a8512f0811efae197'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:53.974633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 913/1819] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:43:53 2022] DELETE /identity/v3/users/2d688d6d8b63472a8512f0811efae197 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:54.277497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:54.279079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:54.292276 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32f400-d5ca-4bb0-b73d-ddae56b940b8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:54.350503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:54.351381 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:54.351663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:54.351936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] PATH_INFO: `/v3/projects/46dbb7100226420fbbf6a8f17ac4acce` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:54.353141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46dbb7100226420fbbf6a8f17ac4acce)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:54.355053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:54.402457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46dbb7100226420fbbf6a8f17ac4acce', 'request_id': 'req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:54.410707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46dbb7100226420fbbf6a8f17ac4acce', 'request_id': 'req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:54.417056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46dbb7100226420fbbf6a8f17ac4acce', 'request_id': 'req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:54.427335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] The token cache is being invalidate because project 46dbb7100226420fbbf6a8f17ac4acce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:43:54.427619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:54.428244 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9efccdfe-8db9-48ce-b6c1-3bfabfe96bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:43:54.429821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 907/1820] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:43:54 2022] DELETE /identity/v3/projects/46dbb7100226420fbbf6a8f17ac4acce => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:54.434856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdbacd-205a-4938-9766-2bce979a7f96 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:54.435379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdbacd-205a-4938-9766-2bce979a7f96 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:54.435379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdbacd-205a-4938-9766-2bce979a7f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:54.455342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-43bdbacd-205a-4938-9766-2bce979a7f96 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:54.456612 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-43bdbacd-205a-4938-9766-2bce979a7f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:54.541387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 914/1821] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:54.543428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cafd5-2666-472e-bbac-2aaa631c6c27 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:54.543724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cafd5-2666-472e-bbac-2aaa631c6c27 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:54.544253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cafd5-2666-472e-bbac-2aaa631c6c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:54.562814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-698cafd5-2666-472e-bbac-2aaa631c6c27 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:54.564030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-698cafd5-2666-472e-bbac-2aaa631c6c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:54.572056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 908/1822] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:54.576469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdbacd-205a-4938-9766-2bce979a7f96 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:54.587054 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdbacd-205a-4938-9766-2bce979a7f96 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:54.591612 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d8d2f-08ee-4025-972c-7e3239fa4015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:54.592516 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d8d2f-08ee-4025-972c-7e3239fa4015 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:54.592905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d8d2f-08ee-4025-972c-7e3239fa4015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:54.593181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d8d2f-08ee-4025-972c-7e3239fa4015 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:54.593910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d8d2f-08ee-4025-972c-7e3239fa4015 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:43:54.594213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d8d2f-08ee-4025-972c-7e3239fa4015 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:54.595802 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d8d2f-08ee-4025-972c-7e3239fa4015 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:54.607410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 915/1823] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:43:54.612357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cafd5-2666-472e-bbac-2aaa631c6c27 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:54.621814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cafd5-2666-472e-bbac-2aaa631c6c27 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:54.627845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999309ba-ee09-40ab-8a4a-e5c935986874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:54.628542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999309ba-ee09-40ab-8a4a-e5c935986874 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:54.628826 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999309ba-ee09-40ab-8a4a-e5c935986874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:54.629102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999309ba-ee09-40ab-8a4a-e5c935986874 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:54.629799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999309ba-ee09-40ab-8a4a-e5c935986874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:54.631693 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999309ba-ee09-40ab-8a4a-e5c935986874 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:54.658285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 909/1824] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:54 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:54.662419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d8d2f-08ee-4025-972c-7e3239fa4015 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:54.672432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d8d2f-08ee-4025-972c-7e3239fa4015 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:54.731068 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ea99a-9017-4c17-8c9b-f02aeec96040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:54.731737 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ea99a-9017-4c17-8c9b-f02aeec96040 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:54.732011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ea99a-9017-4c17-8c9b-f02aeec96040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:54.732279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ea99a-9017-4c17-8c9b-f02aeec96040 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:54.733292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ea99a-9017-4c17-8c9b-f02aeec96040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:54.735105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ea99a-9017-4c17-8c9b-f02aeec96040 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:54.756735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 916/1825] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:54 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:54.760951 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999309ba-ee09-40ab-8a4a-e5c935986874 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:54.772208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999309ba-ee09-40ab-8a4a-e5c935986874 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:54.779987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af79ca7-eff5-43e5-bf58-58c33d762063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:54.779987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af79ca7-eff5-43e5-bf58-58c33d762063 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:54.779987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af79ca7-eff5-43e5-bf58-58c33d762063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:54.779987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af79ca7-eff5-43e5-bf58-58c33d762063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:54.780324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af79ca7-eff5-43e5-bf58-58c33d762063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:54.781561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af79ca7-eff5-43e5-bf58-58c33d762063 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:54.792298 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 910/1826] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:54.797195 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ea99a-9017-4c17-8c9b-f02aeec96040 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:54.813487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ea99a-9017-4c17-8c9b-f02aeec96040 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:54.822759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a66a8-4385-4ac3-bac3-773f24545b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:54.823778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a66a8-4385-4ac3-bac3-773f24545b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:54.824154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a66a8-4385-4ac3-bac3-773f24545b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:54.824546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a66a8-4385-4ac3-bac3-773f24545b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:54.825372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a66a8-4385-4ac3-bac3-773f24545b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:54.828090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a66a8-4385-4ac3-bac3-773f24545b3f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:54.843430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 917/1827] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:54.847712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af79ca7-eff5-43e5-bf58-58c33d762063 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:54.858614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af79ca7-eff5-43e5-bf58-58c33d762063 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:54.864849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3b763-4c96-4ab6-af0f-b37714cf4a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:54.865743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3b763-4c96-4ab6-af0f-b37714cf4a98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:54.866034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3b763-4c96-4ab6-af0f-b37714cf4a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:54.866321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3b763-4c96-4ab6-af0f-b37714cf4a98 admin admin] PATH_INFO: `/v3/projects/a3d5b3e6372a47468a84dd19898bc2f5/users/da0741d0a6d44c649cb0ed19652e81d9/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:54.882252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3b763-4c96-4ab6-af0f-b37714cf4a98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d5b3e6372a47468a84dd19898bc2f5, user_id=da0741d0a6d44c649cb0ed19652e81d9, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:54.884971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3b763-4c96-4ab6-af0f-b37714cf4a98 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:54.901553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 911/1828] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:54 2022] PUT /identity/v3/projects/a3d5b3e6372a47468a84dd19898bc2f5/users/da0741d0a6d44c649cb0ed19652e81d9/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:54.912096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdb636-a6cd-40bc-b35b-a18f1dd00167 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:54.912388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdb636-a6cd-40bc-b35b-a18f1dd00167 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:54.912699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdb636-a6cd-40bc-b35b-a18f1dd00167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:54.939839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c3cdb636-a6cd-40bc-b35b-a18f1dd00167 None None] MFA Rules not processed for user `da0741d0a6d44c649cb0ed19652e81d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:54.941143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c3cdb636-a6cd-40bc-b35b-a18f1dd00167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:55.004857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 918/1829] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:55.009609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3b763-4c96-4ab6-af0f-b37714cf4a98 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.078712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3b763-4c96-4ab6-af0f-b37714cf4a98 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.084677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f9fbb-2e41-423c-908b-1b2bd585db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.085343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f9fbb-2e41-423c-908b-1b2bd585db65 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.085679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f9fbb-2e41-423c-908b-1b2bd585db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.085886 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f9fbb-2e41-423c-908b-1b2bd585db65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.086549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f9fbb-2e41-423c-908b-1b2bd585db65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.088534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f9fbb-2e41-423c-908b-1b2bd585db65 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.116206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 912/1830] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:55 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:55.120117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdb636-a6cd-40bc-b35b-a18f1dd00167 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.131158 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdb636-a6cd-40bc-b35b-a18f1dd00167 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.188918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e119f79-8419-4bc9-b89c-f58d7c6c05ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.189588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119f79-8419-4bc9-b89c-f58d7c6c05ec admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.189862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119f79-8419-4bc9-b89c-f58d7c6c05ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.190129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119f79-8419-4bc9-b89c-f58d7c6c05ec admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.190854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e119f79-8419-4bc9-b89c-f58d7c6c05ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.192808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e119f79-8419-4bc9-b89c-f58d7c6c05ec admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.213455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 919/1831] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:55 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:43:55.217763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f9fbb-2e41-423c-908b-1b2bd585db65 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.228036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f9fbb-2e41-423c-908b-1b2bd585db65 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.233839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897cdc0-d36e-403b-bdee-54be09ad2a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.234500 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897cdc0-d36e-403b-bdee-54be09ad2a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.234773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897cdc0-d36e-403b-bdee-54be09ad2a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.235041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897cdc0-d36e-403b-bdee-54be09ad2a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.235756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897cdc0-d36e-403b-bdee-54be09ad2a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.237385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897cdc0-d36e-403b-bdee-54be09ad2a67 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.248010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 913/1832] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:55.252148 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e119f79-8419-4bc9-b89c-f58d7c6c05ec admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.262700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e119f79-8419-4bc9-b89c-f58d7c6c05ec admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.269017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde059d4-7835-4aec-a997-a8fb5ac322a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.269825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde059d4-7835-4aec-a997-a8fb5ac322a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.270069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde059d4-7835-4aec-a997-a8fb5ac322a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.270448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde059d4-7835-4aec-a997-a8fb5ac322a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.271411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde059d4-7835-4aec-a997-a8fb5ac322a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.273034 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde059d4-7835-4aec-a997-a8fb5ac322a8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.283827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 920/1833] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:55.287697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897cdc0-d36e-403b-bdee-54be09ad2a67 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.299365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897cdc0-d36e-403b-bdee-54be09ad2a67 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.304846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc1665-ed96-4352-8162-6699b4e892e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.305848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc1665-ed96-4352-8162-6699b4e892e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.306430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc1665-ed96-4352-8162-6699b4e892e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.306639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc1665-ed96-4352-8162-6699b4e892e2 admin admin] PATH_INFO: `/v3/projects/e3d635fd5b824c529c5ceaffcff68a0a/users/ca2ad3e7815b4f6989ad54d4c3556748/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.324405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc1665-ed96-4352-8162-6699b4e892e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d635fd5b824c529c5ceaffcff68a0a, user_id=ca2ad3e7815b4f6989ad54d4c3556748, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.327175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc1665-ed96-4352-8162-6699b4e892e2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.342661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 914/1834] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:55 2022] PUT /identity/v3/projects/e3d635fd5b824c529c5ceaffcff68a0a/users/ca2ad3e7815b4f6989ad54d4c3556748/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:55.354058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db2d02-7ee5-4119-8072-5150687d2af5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.354325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db2d02-7ee5-4119-8072-5150687d2af5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.354723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db2d02-7ee5-4119-8072-5150687d2af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.382291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-49db2d02-7ee5-4119-8072-5150687d2af5 None None] MFA Rules not processed for user `ca2ad3e7815b4f6989ad54d4c3556748`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:55.383516 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-49db2d02-7ee5-4119-8072-5150687d2af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:55.448110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 921/1835] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:43:55.453476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc1665-ed96-4352-8162-6699b4e892e2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.515239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc1665-ed96-4352-8162-6699b4e892e2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.520549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39087394-e28c-4abf-9e52-8d87825b7db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.521228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39087394-e28c-4abf-9e52-8d87825b7db6 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.521511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39087394-e28c-4abf-9e52-8d87825b7db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.521783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39087394-e28c-4abf-9e52-8d87825b7db6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.522516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39087394-e28c-4abf-9e52-8d87825b7db6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.524433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39087394-e28c-4abf-9e52-8d87825b7db6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.553701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 915/1836] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:43:55 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:43:55.555733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db2d02-7ee5-4119-8072-5150687d2af5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.566531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db2d02-7ee5-4119-8072-5150687d2af5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.628671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1ca7e-7898-4efc-a177-05f54f871e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.629402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ca7e-7898-4efc-a177-05f54f871e9f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.629687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ca7e-7898-4efc-a177-05f54f871e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.630017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ca7e-7898-4efc-a177-05f54f871e9f admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.630724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1ca7e-7898-4efc-a177-05f54f871e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.632852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1ca7e-7898-4efc-a177-05f54f871e9f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.658494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 922/1837] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:43:55 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:43:55.662433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39087394-e28c-4abf-9e52-8d87825b7db6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.675856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39087394-e28c-4abf-9e52-8d87825b7db6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.680911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05164f-2055-4112-b538-ebea06126250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.681584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05164f-2055-4112-b538-ebea06126250 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.681862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05164f-2055-4112-b538-ebea06126250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.682130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05164f-2055-4112-b538-ebea06126250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.682867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05164f-2055-4112-b538-ebea06126250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.684436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05164f-2055-4112-b538-ebea06126250 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.697580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 916/1838] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:55.701906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1ca7e-7898-4efc-a177-05f54f871e9f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.711939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1ca7e-7898-4efc-a177-05f54f871e9f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.716438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33929f3-fb42-46fb-9b69-07ab7811b4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.717494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33929f3-fb42-46fb-9b69-07ab7811b4b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.717781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33929f3-fb42-46fb-9b69-07ab7811b4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.718054 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33929f3-fb42-46fb-9b69-07ab7811b4b0 admin admin] PATH_INFO: `/v3/projects/6acd6a6ad07f4f678690902ca4c0e969/users/eb1dca305d4c4283b46fceb4b808dfa5/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.733075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33929f3-fb42-46fb-9b69-07ab7811b4b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6acd6a6ad07f4f678690902ca4c0e969, user_id=eb1dca305d4c4283b46fceb4b808dfa5, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.735716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33929f3-fb42-46fb-9b69-07ab7811b4b0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.749594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 923/1839] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:55 2022] PUT /identity/v3/projects/6acd6a6ad07f4f678690902ca4c0e969/users/eb1dca305d4c4283b46fceb4b808dfa5/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:55.753796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05164f-2055-4112-b538-ebea06126250 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.763733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05164f-2055-4112-b538-ebea06126250 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.820700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7f69e-8b78-4625-9570-d1818bbfa4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.821381 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7f69e-8b78-4625-9570-d1818bbfa4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.821657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7f69e-8b78-4625-9570-d1818bbfa4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.821922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7f69e-8b78-4625-9570-d1818bbfa4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.822638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7f69e-8b78-4625-9570-d1818bbfa4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.824335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7f69e-8b78-4625-9570-d1818bbfa4f1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.834923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 917/1840] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:43:55.839233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33929f3-fb42-46fb-9b69-07ab7811b4b0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:55.850063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33929f3-fb42-46fb-9b69-07ab7811b4b0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:55.854882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26360035-ac49-4215-9587-e5067b89beef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:55.855769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26360035-ac49-4215-9587-e5067b89beef admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.856051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26360035-ac49-4215-9587-e5067b89beef admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.856292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26360035-ac49-4215-9587-e5067b89beef admin admin] PATH_INFO: `/v3/projects/6acd6a6ad07f4f678690902ca4c0e969/users/eb1dca305d4c4283b46fceb4b808dfa5/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.859120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26360035-ac49-4215-9587-e5067b89beef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6acd6a6ad07f4f678690902ca4c0e969, user_id=eb1dca305d4c4283b46fceb4b808dfa5, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:55.861717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26360035-ac49-4215-9587-e5067b89beef admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:55.872029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 924/1841] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:43:55 2022] PUT /identity/v3/projects/6acd6a6ad07f4f678690902ca4c0e969/users/eb1dca305d4c4283b46fceb4b808dfa5/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:43:55.882732 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40a319-3e77-4171-bb6e-653f40a737d2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:55.883020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40a319-3e77-4171-bb6e-653f40a737d2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:55.883291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40a319-3e77-4171-bb6e-653f40a737d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:55.908981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-da40a319-3e77-4171-bb6e-653f40a737d2 None None] MFA Rules not processed for user `eb1dca305d4c4283b46fceb4b808dfa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:43:55.910161 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-da40a319-3e77-4171-bb6e-653f40a737d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:55.989358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 918/1842] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:43:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:43:56.005487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26360035-ac49-4215-9587-e5067b89beef admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:56.007949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26360035-ac49-4215-9587-e5067b89beef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:43:56.076517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26360035-ac49-4215-9587-e5067b89beef admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:56.080939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349531a5-ccfd-4784-8fc7-eba0734c957c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:56.081578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349531a5-ccfd-4784-8fc7-eba0734c957c service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:56.081859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349531a5-ccfd-4784-8fc7-eba0734c957c service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:56.082124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349531a5-ccfd-4784-8fc7-eba0734c957c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:56.091379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349531a5-ccfd-4784-8fc7-eba0734c957c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:56.093033 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349531a5-ccfd-4784-8fc7-eba0734c957c service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:56.103458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 925/1843] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:43:56.152178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40a319-3e77-4171-bb6e-653f40a737d2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:43:56.161845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40a319-3e77-4171-bb6e-653f40a737d2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:43:56.165688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdd0ce-8e7e-41eb-9c2e-6ba6f93b4cb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:43:56.166405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd0ce-8e7e-41eb-9c2e-6ba6f93b4cb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:43:56.166669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd0ce-8e7e-41eb-9c2e-6ba6f93b4cb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:43:56.166909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd0ce-8e7e-41eb-9c2e-6ba6f93b4cb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:43:56.222142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdd0ce-8e7e-41eb-9c2e-6ba6f93b4cb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:43:56.223719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdd0ce-8e7e-41eb-9c2e-6ba6f93b4cb9 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:43:56.232739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 919/1844] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:43:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:00.294632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349531a5-ccfd-4784-8fc7-eba0734c957c service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:00.296251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-349531a5-ccfd-4784-8fc7-eba0734c957c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:00.362441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349531a5-ccfd-4784-8fc7-eba0734c957c service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:00.367487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:00.368371 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:00.369192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:00.369476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] PATH_INFO: `/v3/users/fffed55f2973495687fce09a195c76b5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:00.370737 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fffed55f2973495687fce09a195c76b5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:00.372719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:00.418334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fffed55f2973495687fce09a195c76b5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:00.418930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fffed55f2973495687fce09a195c76b5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:00.430937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fffed55f2973495687fce09a195c76b5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:00.439409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fffed55f2973495687fce09a195c76b5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:00.450673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fffed55f2973495687fce09a195c76b5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:00.457921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 926/1845] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:00 2022] DELETE /identity/v3/users/fffed55f2973495687fce09a195c76b5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:00.462680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d4438-859c-42b1-a630-5ab102c8ae47 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:00.463183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d4438-859c-42b1-a630-5ab102c8ae47 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:00.463410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d4438-859c-42b1-a630-5ab102c8ae47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:00.497470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-555d4438-859c-42b1-a630-5ab102c8ae47 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:00.501610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-555d4438-859c-42b1-a630-5ab102c8ae47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:00.571161 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 920/1846] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:00.588781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:00.591214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:00.643893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d4438-859c-42b1-a630-5ab102c8ae47 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:00.649395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-555d4438-859c-42b1-a630-5ab102c8ae47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:00.668050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d4438-859c-42b1-a630-5ab102c8ae47 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:00.674404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:00.675720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:00.676322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:00.676838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] PATH_INFO: `/v3/users/bf050127ee6049e6bf12b303f4972532` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:00.677387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e24266-da60-48ad-a770-f98459979b62 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:00.679072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf050127ee6049e6bf12b303f4972532)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:00.681725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:00.682052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab58e33-92db-441b-a5ca-7e0ddcccc44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:00.682886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58e33-92db-441b-a5ca-7e0ddcccc44c service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:00.683176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58e33-92db-441b-a5ca-7e0ddcccc44c service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:00.683452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58e33-92db-441b-a5ca-7e0ddcccc44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:00.707693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab58e33-92db-441b-a5ca-7e0ddcccc44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:00.709520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab58e33-92db-441b-a5ca-7e0ddcccc44c service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:00.728543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 927/1847] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:00.790544 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf050127ee6049e6bf12b303f4972532'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:00.791443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf050127ee6049e6bf12b303f4972532'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:00.809160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf050127ee6049e6bf12b303f4972532'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:00.828782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf050127ee6049e6bf12b303f4972532'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:00.844113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf050127ee6049e6bf12b303f4972532'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:00.855407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 921/1848] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:00 2022] DELETE /identity/v3/users/bf050127ee6049e6bf12b303f4972532 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:00.860362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2cfe6-61a6-4130-add1-c1fbd400a186 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:00.860885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2cfe6-61a6-4130-add1-c1fbd400a186 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:00.861299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2cfe6-61a6-4130-add1-c1fbd400a186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:00.891437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-79b2cfe6-61a6-4130-add1-c1fbd400a186 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:00.892836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-79b2cfe6-61a6-4130-add1-c1fbd400a186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:00.920460 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:00.922115 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:01.039225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 928/1849] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:01.048933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eda85-68eb-4951-b352-8b107e2f4191 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:01.054806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:01.056439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:01.056824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:01.057198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] PATH_INFO: `/v3/users/da0741d0a6d44c649cb0ed19652e81d9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:01.057431 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2cfe6-61a6-4130-add1-c1fbd400a186 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:01.059120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-79b2cfe6-61a6-4130-add1-c1fbd400a186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:01.059952 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=da0741d0a6d44c649cb0ed19652e81d9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:01.063038 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:01.133713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da0741d0a6d44c649cb0ed19652e81d9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:01.134542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da0741d0a6d44c649cb0ed19652e81d9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:01.151134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2cfe6-61a6-4130-add1-c1fbd400a186 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:01.151455 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da0741d0a6d44c649cb0ed19652e81d9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:01.156202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d28444-e464-46c1-8297-7c670bfa75ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:01.156946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d28444-e464-46c1-8297-7c670bfa75ed service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:01.157398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d28444-e464-46c1-8297-7c670bfa75ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:01.157680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d28444-e464-46c1-8297-7c670bfa75ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:01.159600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f8d28444-e464-46c1-8297-7c670bfa75ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:01.161047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da0741d0a6d44c649cb0ed19652e81d9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:01.170934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-62d832b6-defe-4873-9658-bf26b6789c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da0741d0a6d44c649cb0ed19652e81d9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:01.177179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d28444-e464-46c1-8297-7c670bfa75ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:01.178631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d28444-e464-46c1-8297-7c670bfa75ed service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:01.179533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 922/1850] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:00 2022] DELETE /identity/v3/users/da0741d0a6d44c649cb0ed19652e81d9 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:01.186335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe113a6-fe26-4b55-9858-11a6fdbff55a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:01.187030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe113a6-fe26-4b55-9858-11a6fdbff55a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:01.187030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe113a6-fe26-4b55-9858-11a6fdbff55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:01.190259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 929/1851] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:01.208551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dbe113a6-fe26-4b55-9858-11a6fdbff55a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:01.210015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dbe113a6-fe26-4b55-9858-11a6fdbff55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:01.275998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 923/1852] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:01.298450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d28444-e464-46c1-8297-7c670bfa75ed service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:01.300219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f8d28444-e464-46c1-8297-7c670bfa75ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:01.312913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d28444-e464-46c1-8297-7c670bfa75ed service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:01.387358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5281309-cc81-4358-8215-20dfe69b2139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:01.388257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5281309-cc81-4358-8215-20dfe69b2139 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:01.399260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5281309-cc81-4358-8215-20dfe69b2139 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:01.399885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5281309-cc81-4358-8215-20dfe69b2139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:01.431209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5281309-cc81-4358-8215-20dfe69b2139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:01.432715 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5281309-cc81-4358-8215-20dfe69b2139 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:01.455774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 930/1853] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:44:02.444903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe113a6-fe26-4b55-9858-11a6fdbff55a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:02.447686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dbe113a6-fe26-4b55-9858-11a6fdbff55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:02.461142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe113a6-fe26-4b55-9858-11a6fdbff55a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:02.467406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:02.469942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:02.470443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:02.470951 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] PATH_INFO: `/v3/projects/61de61544d2140e88a73e96e06e3a9b9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:02.472651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] RBAC: Authorizing `identity:delete_project(project_id=61de61544d2140e88a73e96e06e3a9b9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:02.475311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:02.533567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61de61544d2140e88a73e96e06e3a9b9', 'request_id': 'req-54bde1de-85e4-4365-91bd-ffbf060b877f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:02.537977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50127e38-126a-45dd-aaab-0a231f7e55ed None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:02.538517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50127e38-126a-45dd-aaab-0a231f7e55ed None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:02.538517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50127e38-126a-45dd-aaab-0a231f7e55ed None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:02.539784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 931/1854] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:44:02.547523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61de61544d2140e88a73e96e06e3a9b9', 'request_id': 'req-54bde1de-85e4-4365-91bd-ffbf060b877f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:02.555590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7adba-1960-479c-96df-ce12c2489268 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:02.556161 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7adba-1960-479c-96df-ce12c2489268 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:02.556677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7adba-1960-479c-96df-ce12c2489268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:02.565348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61de61544d2140e88a73e96e06e3a9b9', 'request_id': 'req-54bde1de-85e4-4365-91bd-ffbf060b877f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:02.586675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] The token cache is being invalidate because project 61de61544d2140e88a73e96e06e3a9b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:02.587021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:02.587698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:02.607103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 924/1855] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:02 2022] DELETE /identity/v3/projects/61de61544d2140e88a73e96e06e3a9b9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:02.608356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:02.610215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:02.622915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ffe7adba-1960-479c-96df-ce12c2489268 None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:02.624702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ffe7adba-1960-479c-96df-ce12c2489268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:02.775586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bde1de-85e4-4365-91bd-ffbf060b877f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:02.780531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:02.781570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:02.781856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:02.782127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] PATH_INFO: `/v3/users/90c15197eccc4ff4aa128ef4a0bee1bb` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:02.783199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 932/1856] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:44:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:02.783383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c15197eccc4ff4aa128ef4a0bee1bb)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:02.784674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7adba-1960-479c-96df-ce12c2489268 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:02.785391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:02.786397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ffe7adba-1960-479c-96df-ce12c2489268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:02.832603 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7adba-1960-479c-96df-ce12c2489268 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:02.843387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:02.845286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:02.845745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:02.846312 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] PATH_INFO: `/v3/projects/7175d930c72449a3a9ab36f6d9c491d1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:02.848126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7175d930c72449a3a9ab36f6d9c491d1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:02.852849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:02.908050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c15197eccc4ff4aa128ef4a0bee1bb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:02.910547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c15197eccc4ff4aa128ef4a0bee1bb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:02.964023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c15197eccc4ff4aa128ef4a0bee1bb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:02.982111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c15197eccc4ff4aa128ef4a0bee1bb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:02.999778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7175d930c72449a3a9ab36f6d9c491d1', 'request_id': 'req-e318ab0b-42be-47c7-9749-290a420e8147'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.000515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c15197eccc4ff4aa128ef4a0bee1bb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.007796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 925/1857] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:02 2022] DELETE /identity/v3/users/90c15197eccc4ff4aa128ef4a0bee1bb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:03.019069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7175d930c72449a3a9ab36f6d9c491d1', 'request_id': 'req-e318ab0b-42be-47c7-9749-290a420e8147'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.026317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7175d930c72449a3a9ab36f6d9c491d1', 'request_id': 'req-e318ab0b-42be-47c7-9749-290a420e8147'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.042597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] The token cache is being invalidate because project 7175d930c72449a3a9ab36f6d9c491d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:03.043746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.044833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e318ab0b-42be-47c7-9749-290a420e8147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.046922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 933/1858] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:02 2022] DELETE /identity/v3/projects/7175d930c72449a3a9ab36f6d9c491d1 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:03.051524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:03.057066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:03.141226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f99fd-fec9-4763-959c-774a077ada20 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:03.150983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:03.151690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:03.152008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:03.152294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] PATH_INFO: `/v3/users/9bd61a9db1cb44b58e63cfffb1d1e877` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:03.154983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd61a9db1cb44b58e63cfffb1d1e877)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:03.161081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:03.275828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd61a9db1cb44b58e63cfffb1d1e877'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.276365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada03117-8439-4d2e-88b3-f1451bc9fe38 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:03.276730 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada03117-8439-4d2e-88b3-f1451bc9fe38 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:03.276919 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd61a9db1cb44b58e63cfffb1d1e877'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.277017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada03117-8439-4d2e-88b3-f1451bc9fe38 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:03.278614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 934/1859] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:44:03.287579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f3559-e178-4f7b-bd4f-3995456d7132 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:03.288150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f3559-e178-4f7b-bd4f-3995456d7132 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:03.288594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f3559-e178-4f7b-bd4f-3995456d7132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:03.309616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd61a9db1cb44b58e63cfffb1d1e877'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.320662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4d6f3559-e178-4f7b-bd4f-3995456d7132 None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:03.321882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4d6f3559-e178-4f7b-bd4f-3995456d7132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:03.323095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd61a9db1cb44b58e63cfffb1d1e877'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.349302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd61a9db1cb44b58e63cfffb1d1e877'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.376717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 926/1860] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:03 2022] DELETE /identity/v3/users/9bd61a9db1cb44b58e63cfffb1d1e877 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:03.427437 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:03.430100 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:03.480557 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 935/1861] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:44:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:03.481556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f3559-e178-4f7b-bd4f-3995456d7132 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:03.483004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4d6f3559-e178-4f7b-bd4f-3995456d7132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:03.502029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f3559-e178-4f7b-bd4f-3995456d7132 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:03.529754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af032f-14ef-495c-a0f8-360363454a18 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:03.539347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:03.541125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:03.541552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:03.542640 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] PATH_INFO: `/v3/projects/a3d5b3e6372a47468a84dd19898bc2f5` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:03.545191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d5b3e6372a47468a84dd19898bc2f5)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:03.547972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:03.632317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5872b-9c50-421a-b0d8-6ea5121624a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:03.633726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5872b-9c50-421a-b0d8-6ea5121624a9 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:03.634155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5872b-9c50-421a-b0d8-6ea5121624a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:03.634569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5872b-9c50-421a-b0d8-6ea5121624a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:03.638272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-15e5872b-9c50-421a-b0d8-6ea5121624a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:03.645186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d5b3e6372a47468a84dd19898bc2f5', 'request_id': 'req-d9bb24e7-77ec-45f5-9c91-cb3359f76420'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.656848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5872b-9c50-421a-b0d8-6ea5121624a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:03.658574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d5b3e6372a47468a84dd19898bc2f5', 'request_id': 'req-d9bb24e7-77ec-45f5-9c91-cb3359f76420'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.659544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5872b-9c50-421a-b0d8-6ea5121624a9 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:03.667611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d5b3e6372a47468a84dd19898bc2f5', 'request_id': 'req-d9bb24e7-77ec-45f5-9c91-cb3359f76420'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.676395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 936/1862] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:03.688610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] The token cache is being invalidate because project a3d5b3e6372a47468a84dd19898bc2f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:03.688910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.689660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.691701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 927/1863] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:03 2022] DELETE /identity/v3/projects/a3d5b3e6372a47468a84dd19898bc2f5 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:03.695536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5872b-9c50-421a-b0d8-6ea5121624a9 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:03.710799 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5872b-9c50-421a-b0d8-6ea5121624a9 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:03.784089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:03.785013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:03.785291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:03.785616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] PATH_INFO: `/v3/users/ca2ad3e7815b4f6989ad54d4c3556748` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:03.786816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca2ad3e7815b4f6989ad54d4c3556748)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:03.788948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:03.842424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca2ad3e7815b4f6989ad54d4c3556748'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.843289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca2ad3e7815b4f6989ad54d4c3556748'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.857026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca2ad3e7815b4f6989ad54d4c3556748'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.867207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca2ad3e7815b4f6989ad54d4c3556748'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.887692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca2ad3e7815b4f6989ad54d4c3556748'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:03.895204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 937/1864] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:03 2022] DELETE /identity/v3/users/ca2ad3e7815b4f6989ad54d4c3556748 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:04.158213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:04.160468 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.181902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:04.183917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.344090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a3a72-934d-42e5-9919-e5daf988685c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:04.349944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:04.350987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:04.351450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:04.351783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] PATH_INFO: `/v3/projects/06eb6a8f9f9542ff856902d0705409e8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:04.353372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06eb6a8f9f9542ff856902d0705409e8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:04.357625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:04.359060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb24e7-77ec-45f5-9c91-cb3359f76420 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:04.467768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b77b9-354c-4b30-ad72-c431cc770806 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:04.468989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b77b9-354c-4b30-ad72-c431cc770806 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:04.469572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b77b9-354c-4b30-ad72-c431cc770806 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:04.474223 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06eb6a8f9f9542ff856902d0705409e8', 'request_id': 'req-6dccdc3e-1ce4-4580-be18-236bf62c8151'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.480079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b77b9-354c-4b30-ad72-c431cc770806 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:04.482443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cb7b77b9-354c-4b30-ad72-c431cc770806 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.487107 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06eb6a8f9f9542ff856902d0705409e8', 'request_id': 'req-6dccdc3e-1ce4-4580-be18-236bf62c8151'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.496594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b77b9-354c-4b30-ad72-c431cc770806 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:04.498371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b77b9-354c-4b30-ad72-c431cc770806 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:04.508794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06eb6a8f9f9542ff856902d0705409e8', 'request_id': 'req-6dccdc3e-1ce4-4580-be18-236bf62c8151'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.521507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 928/1865] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:04.522544 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b77b9-354c-4b30-ad72-c431cc770806 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:04.526893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cb7b77b9-354c-4b30-ad72-c431cc770806 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.534609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] The token cache is being invalidate because project 06eb6a8f9f9542ff856902d0705409e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:04.535876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.536762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.538811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 938/1866] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:04 2022] DELETE /identity/v3/projects/06eb6a8f9f9542ff856902d0705409e8 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:04.539866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:04.542057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.553777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b77b9-354c-4b30-ad72-c431cc770806 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:04.629852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccdc3e-1ce4-4580-be18-236bf62c8151 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:04.636048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:04.636048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:04.636787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:04.637123 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] PATH_INFO: `/v3/projects/e3d635fd5b824c529c5ceaffcff68a0a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:04.638970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d635fd5b824c529c5ceaffcff68a0a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:04.641146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:04.667378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:04.668407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:04.668731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:04.669011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] PATH_INFO: `/v3/projects/57177c2dacd243508f3c61af94b64496` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:04.670412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57177c2dacd243508f3c61af94b64496)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:04.673359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:04.704396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d635fd5b824c529c5ceaffcff68a0a', 'request_id': 'req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.724711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d635fd5b824c529c5ceaffcff68a0a', 'request_id': 'req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.735076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d635fd5b824c529c5ceaffcff68a0a', 'request_id': 'req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.737546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57177c2dacd243508f3c61af94b64496', 'request_id': 'req-2ef41814-9e68-4d99-a219-5f9e0c7dc314'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.747699 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57177c2dacd243508f3c61af94b64496', 'request_id': 'req-2ef41814-9e68-4d99-a219-5f9e0c7dc314'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.752036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] The token cache is being invalidate because project e3d635fd5b824c529c5ceaffcff68a0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:04.752539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.753415 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c5f0e2e7-c97c-4069-93dd-c0fa1f9ca220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.755584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 939/1867] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:04 2022] DELETE /identity/v3/projects/e3d635fd5b824c529c5ceaffcff68a0a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:04.757134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280471fb-b932-4436-9659-1dd35cba33a3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:04.757520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280471fb-b932-4436-9659-1dd35cba33a3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:04.757920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280471fb-b932-4436-9659-1dd35cba33a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:04.760243 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57177c2dacd243508f3c61af94b64496', 'request_id': 'req-2ef41814-9e68-4d99-a219-5f9e0c7dc314'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.777180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] The token cache is being invalidate because project 57177c2dacd243508f3c61af94b64496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:04.777476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.778294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.785436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 929/1868] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:04 2022] DELETE /identity/v3/projects/57177c2dacd243508f3c61af94b64496 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:04.785436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:04.785436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.785763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-280471fb-b932-4436-9659-1dd35cba33a3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:04.785763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-280471fb-b932-4436-9659-1dd35cba33a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.861553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 940/1869] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:04.863016 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886a64c-e566-48ba-96d6-70a83294bec3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:04.863239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886a64c-e566-48ba-96d6-70a83294bec3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:04.863734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886a64c-e566-48ba-96d6-70a83294bec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:04.875058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef41814-9e68-4d99-a219-5f9e0c7dc314 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:04.881627 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:04.882967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:04.883502 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:04.883779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] PATH_INFO: `/v3/users/eb1dca305d4c4283b46fceb4b808dfa5` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:04.885134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1dca305d4c4283b46fceb4b808dfa5)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:04.887367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:04.887839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2886a64c-e566-48ba-96d6-70a83294bec3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:04.889283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2886a64c-e566-48ba-96d6-70a83294bec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.899351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 941/1870] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:04.900696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d80a3-e79a-483b-b82f-38e32fa682f6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:04.901069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d80a3-e79a-483b-b82f-38e32fa682f6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:04.901448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d80a3-e79a-483b-b82f-38e32fa682f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:04.921029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-762d80a3-e79a-483b-b82f-38e32fa682f6 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:04.922574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-762d80a3-e79a-483b-b82f-38e32fa682f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.931329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 942/1871] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:04.932815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767b8e5-0ffe-4736-a2f2-d1ee1c67648f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:04.933170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767b8e5-0ffe-4736-a2f2-d1ee1c67648f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:04.933507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767b8e5-0ffe-4736-a2f2-d1ee1c67648f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:04.940001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1dca305d4c4283b46fceb4b808dfa5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.941272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1dca305d4c4283b46fceb4b808dfa5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.951505 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f767b8e5-0ffe-4736-a2f2-d1ee1c67648f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:04.953090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1dca305d4c4283b46fceb4b808dfa5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.953487 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f767b8e5-0ffe-4736-a2f2-d1ee1c67648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.959591 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1dca305d4c4283b46fceb4b808dfa5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.961335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 943/1872] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:04.962153 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767b8e5-0ffe-4736-a2f2-d1ee1c67648f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:04.963370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f767b8e5-0ffe-4736-a2f2-d1ee1c67648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:04.967464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1dca305d4c4283b46fceb4b808dfa5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:04.973695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 930/1873] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:04 2022] DELETE /identity/v3/users/eb1dca305d4c4283b46fceb4b808dfa5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:04.974535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:04.974846 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767b8e5-0ffe-4736-a2f2-d1ee1c67648f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.046375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d43314-cdbd-4cdd-a71e-949eb64a3592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.047093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d43314-cdbd-4cdd-a71e-949eb64a3592 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.047436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d43314-cdbd-4cdd-a71e-949eb64a3592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.047709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d43314-cdbd-4cdd-a71e-949eb64a3592 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.048448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d43314-cdbd-4cdd-a71e-949eb64a3592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:44:05.048808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d43314-cdbd-4cdd-a71e-949eb64a3592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.050548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d43314-cdbd-4cdd-a71e-949eb64a3592 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.051853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f79cb3-4189-469c-9f6f-f72511d0f831 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.057379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc527eb-9e4b-4bcc-822e-c05a29c3fd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.058600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc527eb-9e4b-4bcc-822e-c05a29c3fd61 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.058946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc527eb-9e4b-4bcc-822e-c05a29c3fd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.059272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc527eb-9e4b-4bcc-822e-c05a29c3fd61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.060002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc527eb-9e4b-4bcc-822e-c05a29c3fd61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:44:05.060354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc527eb-9e4b-4bcc-822e-c05a29c3fd61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.062459 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc527eb-9e4b-4bcc-822e-c05a29c3fd61 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.066849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 944/1874] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:44:05.067701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d43314-cdbd-4cdd-a71e-949eb64a3592 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.078931 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d43314-cdbd-4cdd-a71e-949eb64a3592 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.079842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 931/1875] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:44:05.084044 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1f701-1838-46be-b4ac-6b7702d989cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.084461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc527eb-9e4b-4bcc-822e-c05a29c3fd61 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.085085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1f701-1838-46be-b4ac-6b7702d989cc admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.085369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1f701-1838-46be-b4ac-6b7702d989cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.085644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1f701-1838-46be-b4ac-6b7702d989cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.086349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1f701-1838-46be-b4ac-6b7702d989cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.088409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1f701-1838-46be-b4ac-6b7702d989cc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.095080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc527eb-9e4b-4bcc-822e-c05a29c3fd61 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.099922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde7574-303f-45ab-a50b-95f96adbb15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.100647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde7574-303f-45ab-a50b-95f96adbb15c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.100926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde7574-303f-45ab-a50b-95f96adbb15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.101526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde7574-303f-45ab-a50b-95f96adbb15c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.102230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde7574-303f-45ab-a50b-95f96adbb15c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.104129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde7574-303f-45ab-a50b-95f96adbb15c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.121399 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 945/1876] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:05.126671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1f701-1838-46be-b4ac-6b7702d989cc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.134108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 932/1877] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:44:05.138424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde7574-303f-45ab-a50b-95f96adbb15c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.209592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1f701-1838-46be-b4ac-6b7702d989cc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.220934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde7574-303f-45ab-a50b-95f96adbb15c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.225574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45a89d-e8ad-4094-82d3-828489fba322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.226259 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45a89d-e8ad-4094-82d3-828489fba322 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.226821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45a89d-e8ad-4094-82d3-828489fba322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.226821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45a89d-e8ad-4094-82d3-828489fba322 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.227493 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45a89d-e8ad-4094-82d3-828489fba322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.229506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45a89d-e8ad-4094-82d3-828489fba322 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.258096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 933/1878] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:05.260108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45a89d-e8ad-4094-82d3-828489fba322 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.275317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafd448-17e3-499e-aebc-23997aa97921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.276005 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafd448-17e3-499e-aebc-23997aa97921 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.276289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafd448-17e3-499e-aebc-23997aa97921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.276593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafd448-17e3-499e-aebc-23997aa97921 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.277289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbafd448-17e3-499e-aebc-23997aa97921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.278835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45a89d-e8ad-4094-82d3-828489fba322 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.279372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbafd448-17e3-499e-aebc-23997aa97921 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.291667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38321827-384d-4d82-a328-746528c54484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.292624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38321827-384d-4d82-a328-746528c54484 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.293364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38321827-384d-4d82-a328-746528c54484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.293757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38321827-384d-4d82-a328-746528c54484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.294539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38321827-384d-4d82-a328-746528c54484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.297154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38321827-384d-4d82-a328-746528c54484 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.306489 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 946/1879] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/users => generated 282 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:05.312819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 934/1880] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:05.313023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafd448-17e3-499e-aebc-23997aa97921 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.315190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38321827-384d-4d82-a328-746528c54484 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.325082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38321827-384d-4d82-a328-746528c54484 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.326781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbafd448-17e3-499e-aebc-23997aa97921 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.330690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860b4ae-c496-407c-a0d1-3c402ac88606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.331790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860b4ae-c496-407c-a0d1-3c402ac88606 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.332135 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860b4ae-c496-407c-a0d1-3c402ac88606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.332434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95993c4-6693-40a1-8424-211360db0289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.332593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860b4ae-c496-407c-a0d1-3c402ac88606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.333271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95993c4-6693-40a1-8424-211360db0289 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.333555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860b4ae-c496-407c-a0d1-3c402ac88606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.333706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95993c4-6693-40a1-8424-211360db0289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.333986 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95993c4-6693-40a1-8424-211360db0289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.334719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95993c4-6693-40a1-8424-211360db0289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.335453 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860b4ae-c496-407c-a0d1-3c402ac88606 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.337255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95993c4-6693-40a1-8424-211360db0289 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.346913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 935/1881] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:44:05.348523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 947/1882] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:05.351338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860b4ae-c496-407c-a0d1-3c402ac88606 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.352423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95993c4-6693-40a1-8424-211360db0289 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.364023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860b4ae-c496-407c-a0d1-3c402ac88606 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.368171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95993c4-6693-40a1-8424-211360db0289 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.370173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a92dd7-c63b-4905-9cd7-d96f089ef84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.371292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92dd7-c63b-4905-9cd7-d96f089ef84b admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.371644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92dd7-c63b-4905-9cd7-d96f089ef84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.371987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92dd7-c63b-4905-9cd7-d96f089ef84b admin admin] PATH_INFO: `/v3/projects/eb2b4debd8b04117bc92a18031123561/users/013fd339e6954911a4568712766f61c0/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.373547 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0703baa-5946-481c-85a6-63252647cd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.374416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0703baa-5946-481c-85a6-63252647cd79 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.374707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0703baa-5946-481c-85a6-63252647cd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.375306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0703baa-5946-481c-85a6-63252647cd79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.376020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0703baa-5946-481c-85a6-63252647cd79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.377736 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0703baa-5946-481c-85a6-63252647cd79 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.391803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a92dd7-c63b-4905-9cd7-d96f089ef84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2b4debd8b04117bc92a18031123561, user_id=013fd339e6954911a4568712766f61c0, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.393462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 948/1883] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:05.394985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a92dd7-c63b-4905-9cd7-d96f089ef84b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.396605 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0703baa-5946-481c-85a6-63252647cd79 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.409330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0703baa-5946-481c-85a6-63252647cd79 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.429304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 936/1884] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:05 2022] PUT /identity/v3/projects/eb2b4debd8b04117bc92a18031123561/users/013fd339e6954911a4568712766f61c0/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:05.432545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f0b36-7af0-4e2e-84c7-f50a3a949347 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.432835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f0b36-7af0-4e2e-84c7-f50a3a949347 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.433400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f0b36-7af0-4e2e-84c7-f50a3a949347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.476935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223701b9-0dc1-4916-b92a-38589940eed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.477837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223701b9-0dc1-4916-b92a-38589940eed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.478113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223701b9-0dc1-4916-b92a-38589940eed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.478380 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223701b9-0dc1-4916-b92a-38589940eed5 admin admin] PATH_INFO: `/v3/projects/88805626f93d40f3b223d72c85eda4a3/users/74825466a5f74d4eb15b5f76bc02ffe7/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.478886 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1a1f0b36-7af0-4e2e-84c7-f50a3a949347 None None] MFA Rules not processed for user `013fd339e6954911a4568712766f61c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:05.480139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1a1f0b36-7af0-4e2e-84c7-f50a3a949347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:05.493801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223701b9-0dc1-4916-b92a-38589940eed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88805626f93d40f3b223d72c85eda4a3, user_id=74825466a5f74d4eb15b5f76bc02ffe7, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.496383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223701b9-0dc1-4916-b92a-38589940eed5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.513143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 949/1885] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:05 2022] PUT /identity/v3/projects/88805626f93d40f3b223d72c85eda4a3/users/74825466a5f74d4eb15b5f76bc02ffe7/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:05.524221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abafdf-edb4-4dbf-b31c-8d32feeddbc2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.524562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abafdf-edb4-4dbf-b31c-8d32feeddbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.524809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abafdf-edb4-4dbf-b31c-8d32feeddbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.551148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 937/1886] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:05.555718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f0b36-7af0-4e2e-84c7-f50a3a949347 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.558300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-89abafdf-edb4-4dbf-b31c-8d32feeddbc2 None None] MFA Rules not processed for user `74825466a5f74d4eb15b5f76bc02ffe7`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:05.559669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-89abafdf-edb4-4dbf-b31c-8d32feeddbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:05.567713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f0b36-7af0-4e2e-84c7-f50a3a949347 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.643728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 950/1887] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:05.644678 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d5fdd-82d4-41fe-8e32-ad9f89f2e1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.644827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abafdf-edb4-4dbf-b31c-8d32feeddbc2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.645731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d5fdd-82d4-41fe-8e32-ad9f89f2e1a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.646075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d5fdd-82d4-41fe-8e32-ad9f89f2e1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.646407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d5fdd-82d4-41fe-8e32-ad9f89f2e1a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.647234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d5fdd-82d4-41fe-8e32-ad9f89f2e1a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.647896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-89abafdf-edb4-4dbf-b31c-8d32feeddbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:05.649293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d5fdd-82d4-41fe-8e32-ad9f89f2e1a4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.663425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abafdf-edb4-4dbf-b31c-8d32feeddbc2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.668683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.669579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.669877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.670193 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] PATH_INFO: `/v3/projects/6acd6a6ad07f4f678690902ca4c0e969` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.671741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6acd6a6ad07f4f678690902ca4c0e969)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.673679 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.679436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 938/1888] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:44:05.679436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d5fdd-82d4-41fe-8e32-ad9f89f2e1a4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.722878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6acd6a6ad07f4f678690902ca4c0e969', 'request_id': 'req-cb308c77-de7a-43f6-9a60-c116eea2de4b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:05.731877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6acd6a6ad07f4f678690902ca4c0e969', 'request_id': 'req-cb308c77-de7a-43f6-9a60-c116eea2de4b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:05.738288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6acd6a6ad07f4f678690902ca4c0e969', 'request_id': 'req-cb308c77-de7a-43f6-9a60-c116eea2de4b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:05.744296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d5fdd-82d4-41fe-8e32-ad9f89f2e1a4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.749671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] The token cache is being invalidate because project 6acd6a6ad07f4f678690902ca4c0e969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:05.749969 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:05.750696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:05.752544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 951/1889] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:05 2022] DELETE /identity/v3/projects/6acd6a6ad07f4f678690902ca4c0e969 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:05.753215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.754776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:05.809763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955a12d-883a-4d6d-a244-15022aa227bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.810429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955a12d-883a-4d6d-a244-15022aa227bf admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.810718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955a12d-883a-4d6d-a244-15022aa227bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.810987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955a12d-883a-4d6d-a244-15022aa227bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.812063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955a12d-883a-4d6d-a244-15022aa227bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.814064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955a12d-883a-4d6d-a244-15022aa227bf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.823097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb308c77-de7a-43f6-9a60-c116eea2de4b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.828338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7797c2a-9192-4efc-8cc3-cf5bb271b1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.829167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7797c2a-9192-4efc-8cc3-cf5bb271b1c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.829589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7797c2a-9192-4efc-8cc3-cf5bb271b1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.830212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7797c2a-9192-4efc-8cc3-cf5bb271b1c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.830574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7797c2a-9192-4efc-8cc3-cf5bb271b1c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.832458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7797c2a-9192-4efc-8cc3-cf5bb271b1c4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.843144 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 939/1890] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:05.844516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0403b-d341-45f1-8d8f-c147c5ab32b4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.844821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0403b-d341-45f1-8d8f-c147c5ab32b4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.845103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0403b-d341-45f1-8d8f-c147c5ab32b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.867531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 952/1891] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:05.869170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7797c2a-9192-4efc-8cc3-cf5bb271b1c4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.870983 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7797c2a-9192-4efc-8cc3-cf5bb271b1c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:05.875392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-82b0403b-d341-45f1-8d8f-c147c5ab32b4 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:05.876368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-82b0403b-d341-45f1-8d8f-c147c5ab32b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:05.887948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7797c2a-9192-4efc-8cc3-cf5bb271b1c4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.956112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 940/1892] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:05.957024 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0403b-d341-45f1-8d8f-c147c5ab32b4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:05.966120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0403b-d341-45f1-8d8f-c147c5ab32b4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:05.970578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e48df8-c505-4306-8d1d-14de10692703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.971538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e48df8-c505-4306-8d1d-14de10692703 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.971744 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b3249-d2f6-425e-82a7-da21de268932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:05.972216 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e48df8-c505-4306-8d1d-14de10692703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.972408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b3249-d2f6-425e-82a7-da21de268932 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.973142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b3249-d2f6-425e-82a7-da21de268932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.973248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e48df8-c505-4306-8d1d-14de10692703 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.973328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b3249-d2f6-425e-82a7-da21de268932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:05.973704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b3249-d2f6-425e-82a7-da21de268932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.973794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e48df8-c505-4306-8d1d-14de10692703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:05.975307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b3249-d2f6-425e-82a7-da21de268932 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.975871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e48df8-c505-4306-8d1d-14de10692703 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:05.990618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 941/1893] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:05.990618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071600b5-8962-489d-9b06-ac05711ba202 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:05.990618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071600b5-8962-489d-9b06-ac05711ba202 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:05.990618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071600b5-8962-489d-9b06-ac05711ba202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.002123 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 953/1894] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:06.003036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e48df8-c505-4306-8d1d-14de10692703 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.014729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e48df8-c505-4306-8d1d-14de10692703 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.016687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-071600b5-8962-489d-9b06-ac05711ba202 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:06.017892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-071600b5-8962-489d-9b06-ac05711ba202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:06.024252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca4b7e-9e3c-4615-8527-2b79eb548c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.025120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca4b7e-9e3c-4615-8527-2b79eb548c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.025563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca4b7e-9e3c-4615-8527-2b79eb548c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.025771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca4b7e-9e3c-4615-8527-2b79eb548c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.026490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca4b7e-9e3c-4615-8527-2b79eb548c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.027543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 942/1895] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:06.028148 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca4b7e-9e3c-4615-8527-2b79eb548c30 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.028299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071600b5-8962-489d-9b06-ac05711ba202 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.039608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 954/1896] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:06.040473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca4b7e-9e3c-4615-8527-2b79eb548c30 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.042322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071600b5-8962-489d-9b06-ac05711ba202 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.047077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bb4dd-1fdb-4622-8b50-8ac0dd7d6d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.047785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bb4dd-1fdb-4622-8b50-8ac0dd7d6d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.048065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bb4dd-1fdb-4622-8b50-8ac0dd7d6d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.048339 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bb4dd-1fdb-4622-8b50-8ac0dd7d6d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.049696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bb4dd-1fdb-4622-8b50-8ac0dd7d6d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.051305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bb4dd-1fdb-4622-8b50-8ac0dd7d6d97 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.052289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca4b7e-9e3c-4615-8527-2b79eb548c30 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.056790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cccfc-0948-4e25-84fe-bb90bc131f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.057432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cccfc-0948-4e25-84fe-bb90bc131f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.057714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cccfc-0948-4e25-84fe-bb90bc131f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.057992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cccfc-0948-4e25-84fe-bb90bc131f3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.058709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cccfc-0948-4e25-84fe-bb90bc131f3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:44:06.059014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cccfc-0948-4e25-84fe-bb90bc131f3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.060611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cccfc-0948-4e25-84fe-bb90bc131f3e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.061177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 943/1897] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:06.061999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bb4dd-1fdb-4622-8b50-8ac0dd7d6d97 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.073029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bb4dd-1fdb-4622-8b50-8ac0dd7d6d97 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.073781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 955/1898] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:44:06.074350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cccfc-0948-4e25-84fe-bb90bc131f3e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.078352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc224781-391b-42f9-b358-703028146bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.079239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc224781-391b-42f9-b358-703028146bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.079522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc224781-391b-42f9-b358-703028146bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.079797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc224781-391b-42f9-b358-703028146bf6 admin admin] PATH_INFO: `/v3/projects/a1b6f1948bf14983845bfe6126bfc4e3/users/7625056b8b6740fd8586940a54178a2c/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.085050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cccfc-0948-4e25-84fe-bb90bc131f3e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.090259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6632c63-e205-4478-810c-7091dbb42480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.091136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6632c63-e205-4478-810c-7091dbb42480 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.091421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6632c63-e205-4478-810c-7091dbb42480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.091702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6632c63-e205-4478-810c-7091dbb42480 admin admin] PATH_INFO: `/v3/projects/2ade947051be468ca93368f65b4973a6/users/9231880c82a847f2b23b3658bb8e1a54/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.094962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc224781-391b-42f9-b358-703028146bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b6f1948bf14983845bfe6126bfc4e3, user_id=7625056b8b6740fd8586940a54178a2c, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.097676 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc224781-391b-42f9-b358-703028146bf6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.106673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6632c63-e205-4478-810c-7091dbb42480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ade947051be468ca93368f65b4973a6, user_id=9231880c82a847f2b23b3658bb8e1a54, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.110096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6632c63-e205-4478-810c-7091dbb42480 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.111899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 944/1899] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:06 2022] PUT /identity/v3/projects/a1b6f1948bf14983845bfe6126bfc4e3/users/7625056b8b6740fd8586940a54178a2c/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:06.113097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc224781-391b-42f9-b358-703028146bf6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.130271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 956/1900] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:06 2022] PUT /identity/v3/projects/2ade947051be468ca93368f65b4973a6/users/9231880c82a847f2b23b3658bb8e1a54/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:06.131553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e928feb-7a18-4111-80cf-620969730b5b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.131837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e928feb-7a18-4111-80cf-620969730b5b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.132109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e928feb-7a18-4111-80cf-620969730b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.157466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0e928feb-7a18-4111-80cf-620969730b5b None None] MFA Rules not processed for user `7625056b8b6740fd8586940a54178a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:06.158672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0e928feb-7a18-4111-80cf-620969730b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:06.181026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc224781-391b-42f9-b358-703028146bf6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.223058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 957/1901] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:06.224144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e928feb-7a18-4111-80cf-620969730b5b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.235003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a34bd5-47ac-413a-8648-634c19faa281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.235704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a34bd5-47ac-413a-8648-634c19faa281 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.235984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a34bd5-47ac-413a-8648-634c19faa281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.236249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a34bd5-47ac-413a-8648-634c19faa281 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.236956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a34bd5-47ac-413a-8648-634c19faa281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.238909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a34bd5-47ac-413a-8648-634c19faa281 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.265145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 945/1902] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:06 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:06.265950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a34bd5-47ac-413a-8648-634c19faa281 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.291700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e928feb-7a18-4111-80cf-620969730b5b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.324520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a34bd5-47ac-413a-8648-634c19faa281 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.330619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e729a7d-5f87-4af6-9e94-5f8aad8cc185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.331284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729a7d-5f87-4af6-9e94-5f8aad8cc185 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.331566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729a7d-5f87-4af6-9e94-5f8aad8cc185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.331849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729a7d-5f87-4af6-9e94-5f8aad8cc185 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.332735 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e729a7d-5f87-4af6-9e94-5f8aad8cc185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.334654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e729a7d-5f87-4af6-9e94-5f8aad8cc185 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.353038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807a284-959b-41c3-b9ba-e6fabe1fd662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.353744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807a284-959b-41c3-b9ba-e6fabe1fd662 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.354060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807a284-959b-41c3-b9ba-e6fabe1fd662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.354360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807a284-959b-41c3-b9ba-e6fabe1fd662 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.355217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807a284-959b-41c3-b9ba-e6fabe1fd662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.357023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807a284-959b-41c3-b9ba-e6fabe1fd662 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.372301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 958/1903] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:06.372427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 946/1904] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:06 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:06.372648 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807a284-959b-41c3-b9ba-e6fabe1fd662 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.372848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e729a7d-5f87-4af6-9e94-5f8aad8cc185 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.379421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807a284-959b-41c3-b9ba-e6fabe1fd662 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.430173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e729a7d-5f87-4af6-9e94-5f8aad8cc185 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.434048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f7fc2-2a5f-4dba-b7a2-aebda010e945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.434669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f7fc2-2a5f-4dba-b7a2-aebda010e945 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.434946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f7fc2-2a5f-4dba-b7a2-aebda010e945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.435133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d5931-6263-4f55-b0c8-cf4ce915679b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.435242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f7fc2-2a5f-4dba-b7a2-aebda010e945 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.435799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d5931-6263-4f55-b0c8-cf4ce915679b admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.435896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f7fc2-2a5f-4dba-b7a2-aebda010e945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.436078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d5931-6263-4f55-b0c8-cf4ce915679b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.436350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d5931-6263-4f55-b0c8-cf4ce915679b admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.437068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d5931-6263-4f55-b0c8-cf4ce915679b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.437949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f7fc2-2a5f-4dba-b7a2-aebda010e945 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.438953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d5931-6263-4f55-b0c8-cf4ce915679b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.459372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 959/1905] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:06 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:06.460163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f7fc2-2a5f-4dba-b7a2-aebda010e945 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.463044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 947/1906] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:06 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:06.463929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d5931-6263-4f55-b0c8-cf4ce915679b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.473058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f7fc2-2a5f-4dba-b7a2-aebda010e945 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.473427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d5931-6263-4f55-b0c8-cf4ce915679b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.478684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d56ab-3593-49ad-9387-484adb99b8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.479044 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f63a51-cb48-446c-9c6a-148d94a003c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.479435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d56ab-3593-49ad-9387-484adb99b8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.479714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d56ab-3593-49ad-9387-484adb99b8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.479983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d56ab-3593-49ad-9387-484adb99b8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.480790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f63a51-cb48-446c-9c6a-148d94a003c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.480944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d56ab-3593-49ad-9387-484adb99b8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.481111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f63a51-cb48-446c-9c6a-148d94a003c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.481369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f63a51-cb48-446c-9c6a-148d94a003c8 admin admin] PATH_INFO: `/v3/projects/2ade947051be468ca93368f65b4973a6/users/9231880c82a847f2b23b3658bb8e1a54/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.482497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d56ab-3593-49ad-9387-484adb99b8b0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.484196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f63a51-cb48-446c-9c6a-148d94a003c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ade947051be468ca93368f65b4973a6, user_id=9231880c82a847f2b23b3658bb8e1a54, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.487474 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f63a51-cb48-446c-9c6a-148d94a003c8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.495439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 948/1907] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:06.495439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d56ab-3593-49ad-9387-484adb99b8b0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.501530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 960/1908] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:06 2022] PUT /identity/v3/projects/2ade947051be468ca93368f65b4973a6/users/9231880c82a847f2b23b3658bb8e1a54/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:06.502507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f63a51-cb48-446c-9c6a-148d94a003c8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.506118 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d56ab-3593-49ad-9387-484adb99b8b0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.565325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf709b05-11be-4f46-94b9-42829bdb3c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.566067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf709b05-11be-4f46-94b9-42829bdb3c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.566409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf709b05-11be-4f46-94b9-42829bdb3c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.566768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf709b05-11be-4f46-94b9-42829bdb3c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.567555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf709b05-11be-4f46-94b9-42829bdb3c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.568640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f63a51-cb48-446c-9c6a-148d94a003c8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.569361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf709b05-11be-4f46-94b9-42829bdb3c35 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.574136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5ce81-fc85-419d-b9d3-2e61f03e693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.574810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5ce81-fc85-419d-b9d3-2e61f03e693f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.575062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5ce81-fc85-419d-b9d3-2e61f03e693f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.575653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5ce81-fc85-419d-b9d3-2e61f03e693f admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.576157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5ce81-fc85-419d-b9d3-2e61f03e693f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.579086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5ce81-fc85-419d-b9d3-2e61f03e693f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.582564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 949/1909] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:06.583966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30522a-3110-4875-9d3d-4a2ad401100d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.584320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30522a-3110-4875-9d3d-4a2ad401100d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.584679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30522a-3110-4875-9d3d-4a2ad401100d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.593127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 961/1910] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:06.593127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5ce81-fc85-419d-b9d3-2e61f03e693f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.604652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5ce81-fc85-419d-b9d3-2e61f03e693f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.611376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dd4db-4322-4d85-add6-6c3496ee9416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.612395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dd4db-4322-4d85-add6-6c3496ee9416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.614155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dd4db-4322-4d85-add6-6c3496ee9416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.614155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dd4db-4322-4d85-add6-6c3496ee9416 admin admin] PATH_INFO: `/v3/projects/9f4978e0135a482e8c85a72cc0781e94/users/5c885adc357c4c669af20fe5ed4faf8d/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.626203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7f30522a-3110-4875-9d3d-4a2ad401100d None None] MFA Rules not processed for user `9231880c82a847f2b23b3658bb8e1a54`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:06.629854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7f30522a-3110-4875-9d3d-4a2ad401100d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:06.641705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6dd4db-4322-4d85-add6-6c3496ee9416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4978e0135a482e8c85a72cc0781e94, user_id=5c885adc357c4c669af20fe5ed4faf8d, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.645329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6dd4db-4322-4d85-add6-6c3496ee9416 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.663575 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 962/1911] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:06 2022] PUT /identity/v3/projects/9f4978e0135a482e8c85a72cc0781e94/users/5c885adc357c4c669af20fe5ed4faf8d/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:06.664116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dd4db-4322-4d85-add6-6c3496ee9416 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.706472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 950/1912] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:06.709749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30522a-3110-4875-9d3d-4a2ad401100d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.724858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30522a-3110-4875-9d3d-4a2ad401100d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.729686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ad168-6c96-4ff0-b307-9e63bdcc0653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.730364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ad168-6c96-4ff0-b307-9e63bdcc0653 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.730643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ad168-6c96-4ff0-b307-9e63bdcc0653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.730916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ad168-6c96-4ff0-b307-9e63bdcc0653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.731642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ad168-6c96-4ff0-b307-9e63bdcc0653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.733280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ad168-6c96-4ff0-b307-9e63bdcc0653 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.733422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dd4db-4322-4d85-add6-6c3496ee9416 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.738858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062a2ee-ccf7-4640-b55e-fbbc84827f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.739698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062a2ee-ccf7-4640-b55e-fbbc84827f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.739981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062a2ee-ccf7-4640-b55e-fbbc84827f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.740306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062a2ee-ccf7-4640-b55e-fbbc84827f69 admin admin] PATH_INFO: `/v3/projects/752a506f89944f5e81c157c956f3884d/users/5cd245168bbd48048965cf030b47bc3c/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.743491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 951/1913] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:06.747742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ad168-6c96-4ff0-b307-9e63bdcc0653 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.749658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8e7ad168-6c96-4ff0-b307-9e63bdcc0653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:06.757927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062a2ee-ccf7-4640-b55e-fbbc84827f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=752a506f89944f5e81c157c956f3884d, user_id=5cd245168bbd48048965cf030b47bc3c, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.760735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062a2ee-ccf7-4640-b55e-fbbc84827f69 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.778210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 963/1914] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:06 2022] PUT /identity/v3/projects/752a506f89944f5e81c157c956f3884d/users/5cd245168bbd48048965cf030b47bc3c/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:06.779360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062a2ee-ccf7-4640-b55e-fbbc84827f69 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.819284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ad168-6c96-4ff0-b307-9e63bdcc0653 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.847690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062a2ee-ccf7-4640-b55e-fbbc84827f69 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:06.852463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f8bdd-c3d6-4b15-8e17-c728e9856673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.853384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f8bdd-c3d6-4b15-8e17-c728e9856673 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.853665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f8bdd-c3d6-4b15-8e17-c728e9856673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.853945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f8bdd-c3d6-4b15-8e17-c728e9856673 admin admin] PATH_INFO: `/v3/projects/9f4978e0135a482e8c85a72cc0781e94/users/5c885adc357c4c669af20fe5ed4faf8d/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.855960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f8bdd-c3d6-4b15-8e17-c728e9856673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4978e0135a482e8c85a72cc0781e94, user_id=5c885adc357c4c669af20fe5ed4faf8d, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.858703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f8bdd-c3d6-4b15-8e17-c728e9856673 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.867000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c248623-e400-4912-980a-7269e2a690a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:06.867650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c248623-e400-4912-980a-7269e2a690a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.867929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c248623-e400-4912-980a-7269e2a690a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.868354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c248623-e400-4912-980a-7269e2a690a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.869377 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 964/1915] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:06 2022] PUT /identity/v3/projects/9f4978e0135a482e8c85a72cc0781e94/users/5c885adc357c4c669af20fe5ed4faf8d/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:06.870594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36981-e579-4c0f-a886-476cd0c5cd18 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.870878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36981-e579-4c0f-a886-476cd0c5cd18 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.871185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36981-e579-4c0f-a886-476cd0c5cd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.898832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6ec36981-e579-4c0f-a886-476cd0c5cd18 None None] MFA Rules not processed for user `5cd245168bbd48048965cf030b47bc3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:06.900050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ec36981-e579-4c0f-a886-476cd0c5cd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:06.942672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c248623-e400-4912-980a-7269e2a690a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:06.944123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c248623-e400-4912-980a-7269e2a690a6 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:06.969017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 952/1916] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:06.970555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62965e-a1d5-427f-bffd-be621f202e57 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:06.970846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62965e-a1d5-427f-bffd-be621f202e57 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:06.971125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62965e-a1d5-427f-bffd-be621f202e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:06.979577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 965/1917] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:06.991665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36981-e579-4c0f-a886-476cd0c5cd18 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:06.993554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ec36981-e579-4c0f-a886-476cd0c5cd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:07.013303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8a62965e-a1d5-427f-bffd-be621f202e57 None None] MFA Rules not processed for user `5c885adc357c4c669af20fe5ed4faf8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:07.014507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8a62965e-a1d5-427f-bffd-be621f202e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:07.088948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36981-e579-4c0f-a886-476cd0c5cd18 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:07.096247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714a32a-5c96-4ba6-9bc8-cc587db9e2e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:07.097705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714a32a-5c96-4ba6-9bc8-cc587db9e2e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:07.100980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714a32a-5c96-4ba6-9bc8-cc587db9e2e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:07.105976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714a32a-5c96-4ba6-9bc8-cc587db9e2e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:07.106204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 953/1918] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:07.118380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62965e-a1d5-427f-bffd-be621f202e57 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:07.124976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714a32a-5c96-4ba6-9bc8-cc587db9e2e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:07.126422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714a32a-5c96-4ba6-9bc8-cc587db9e2e1 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:07.128146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62965e-a1d5-427f-bffd-be621f202e57 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:07.133142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc3b6a-bd89-48ba-ba58-a62579145d4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:07.133800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc3b6a-bd89-48ba-ba58-a62579145d4c service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:07.134081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc3b6a-bd89-48ba-ba58-a62579145d4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:07.134356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc3b6a-bd89-48ba-ba58-a62579145d4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:07.136150 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c4cc3b6a-bd89-48ba-ba58-a62579145d4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:07.137424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 966/1919] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:07.157550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714a32a-5c96-4ba6-9bc8-cc587db9e2e1 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:07.207081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc3b6a-bd89-48ba-ba58-a62579145d4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:07.209014 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc3b6a-bd89-48ba-ba58-a62579145d4c service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:07.222788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 954/1920] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:07.238097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714a32a-5c96-4ba6-9bc8-cc587db9e2e1 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:07.244044 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:07.245113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:07.245384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:07.245702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] PATH_INFO: `/v3/users/5cd245168bbd48048965cf030b47bc3c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:07.246922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd245168bbd48048965cf030b47bc3c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:07.250095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:07.315691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd245168bbd48048965cf030b47bc3c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:07.316834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd245168bbd48048965cf030b47bc3c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:07.334774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd245168bbd48048965cf030b47bc3c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:07.345442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd245168bbd48048965cf030b47bc3c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:07.356817 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd245168bbd48048965cf030b47bc3c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:07.366865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 967/1921] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:07 2022] DELETE /identity/v3/users/5cd245168bbd48048965cf030b47bc3c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:07.372833 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d90ce-0204-4b19-9e24-f40127b23075 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:07.373162 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d90ce-0204-4b19-9e24-f40127b23075 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:07.373532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d90ce-0204-4b19-9e24-f40127b23075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:07.395032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9c0d90ce-0204-4b19-9e24-f40127b23075 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:07.395319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9c0d90ce-0204-4b19-9e24-f40127b23075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:07.467620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 955/1922] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:07.474548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:07.476042 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:07.551611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c128d52-53c3-42db-ab58-7af9c3d2308c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:07.556596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c2447-8b8f-4d8d-bbeb-02c7c1681161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:07.557409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c2447-8b8f-4d8d-bbeb-02c7c1681161 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:07.557679 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c2447-8b8f-4d8d-bbeb-02c7c1681161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:07.557951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c2447-8b8f-4d8d-bbeb-02c7c1681161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:07.562636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d90ce-0204-4b19-9e24-f40127b23075 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:07.564450 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9c0d90ce-0204-4b19-9e24-f40127b23075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:07.571017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c2447-8b8f-4d8d-bbeb-02c7c1681161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:07.573255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c2447-8b8f-4d8d-bbeb-02c7c1681161 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:07.578457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d90ce-0204-4b19-9e24-f40127b23075 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:07.584059 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:07.585307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:07.585665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:07.586087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] PATH_INFO: `/v3/users/74825466a5f74d4eb15b5f76bc02ffe7` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:07.587914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=74825466a5f74d4eb15b5f76bc02ffe7)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:07.589936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:07.593304 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 968/1923] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:07.666483 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74825466a5f74d4eb15b5f76bc02ffe7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:07.667384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74825466a5f74d4eb15b5f76bc02ffe7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:07.682643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74825466a5f74d4eb15b5f76bc02ffe7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:07.690837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74825466a5f74d4eb15b5f76bc02ffe7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:07.698590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74825466a5f74d4eb15b5f76bc02ffe7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:07.705944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 956/1924] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:07 2022] DELETE /identity/v3/users/74825466a5f74d4eb15b5f76bc02ffe7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:07.710726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b78440-89db-442c-8264-fa7b76ea53a4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:07.711246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b78440-89db-442c-8264-fa7b76ea53a4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:07.711403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b78440-89db-442c-8264-fa7b76ea53a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:07.728889 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-51b78440-89db-442c-8264-fa7b76ea53a4 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:07.730093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-51b78440-89db-442c-8264-fa7b76ea53a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:07.790264 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 969/1925] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:07.801412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:07.803045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:07.877330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8c7b3-48a3-44bd-9590-3cce8a252fac admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:07.881438 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847ddb6-800e-4059-9650-f8c0d592b5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:07.882095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847ddb6-800e-4059-9650-f8c0d592b5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:07.882371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847ddb6-800e-4059-9650-f8c0d592b5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:07.882637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847ddb6-800e-4059-9650-f8c0d592b5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:07.891851 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847ddb6-800e-4059-9650-f8c0d592b5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:07.893326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847ddb6-800e-4059-9650-f8c0d592b5d7 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:07.902877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 957/1926] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:08.033167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b78440-89db-442c-8264-fa7b76ea53a4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.034754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-51b78440-89db-442c-8264-fa7b76ea53a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:08.047885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b78440-89db-442c-8264-fa7b76ea53a4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:08.057298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:08.058291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.058496 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.058773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] PATH_INFO: `/v3/projects/752a506f89944f5e81c157c956f3884d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.060015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=752a506f89944f5e81c157c956f3884d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:08.062755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:08.114734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '752a506f89944f5e81c157c956f3884d', 'request_id': 'req-168ab674-7614-4903-980e-808b3c465fc7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.126276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '752a506f89944f5e81c157c956f3884d', 'request_id': 'req-168ab674-7614-4903-980e-808b3c465fc7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.133146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '752a506f89944f5e81c157c956f3884d', 'request_id': 'req-168ab674-7614-4903-980e-808b3c465fc7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.145937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] The token cache is being invalidate because project 752a506f89944f5e81c157c956f3884d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:08.146243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.147056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-168ab674-7614-4903-980e-808b3c465fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.148714 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 970/1927] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:08 2022] DELETE /identity/v3/projects/752a506f89944f5e81c157c956f3884d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:08.157055 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adf8a5-84f5-446a-9fcd-ed4688ea22eb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.157404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adf8a5-84f5-446a-9fcd-ed4688ea22eb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.157623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adf8a5-84f5-446a-9fcd-ed4688ea22eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.178628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b5adf8a5-84f5-446a-9fcd-ed4688ea22eb None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:08.179835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b5adf8a5-84f5-446a-9fcd-ed4688ea22eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:08.238388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 958/1928] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:08.245987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52654c-76fc-4505-a50c-3867a4c3f7d1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.246246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52654c-76fc-4505-a50c-3867a4c3f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.246563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52654c-76fc-4505-a50c-3867a4c3f7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.266257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bf52654c-76fc-4505-a50c-3867a4c3f7d1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:08.267443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bf52654c-76fc-4505-a50c-3867a4c3f7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:08.276249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 971/1929] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:08.280434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adf8a5-84f5-446a-9fcd-ed4688ea22eb None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.289804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adf8a5-84f5-446a-9fcd-ed4688ea22eb None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:08.294720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3401b79-8ce5-4951-b3c3-29ecdaea1faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:08.295374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3401b79-8ce5-4951-b3c3-29ecdaea1faa admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.295633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3401b79-8ce5-4951-b3c3-29ecdaea1faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.296051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3401b79-8ce5-4951-b3c3-29ecdaea1faa admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.296815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3401b79-8ce5-4951-b3c3-29ecdaea1faa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:44:08.297122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3401b79-8ce5-4951-b3c3-29ecdaea1faa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:08.298716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3401b79-8ce5-4951-b3c3-29ecdaea1faa admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:08.318497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 959/1930] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:44:08.322309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52654c-76fc-4505-a50c-3867a4c3f7d1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.334942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52654c-76fc-4505-a50c-3867a4c3f7d1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:08.341140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b8f8e-8781-4ebc-9340-32e015cb0e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:08.341809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b8f8e-8781-4ebc-9340-32e015cb0e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.342092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b8f8e-8781-4ebc-9340-32e015cb0e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.342363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b8f8e-8781-4ebc-9340-32e015cb0e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.343284 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b8f8e-8781-4ebc-9340-32e015cb0e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:08.345911 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b8f8e-8781-4ebc-9340-32e015cb0e26 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:08.363049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3401b79-8ce5-4951-b3c3-29ecdaea1faa admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.364885 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d3401b79-8ce5-4951-b3c3-29ecdaea1faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:08.389446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 972/1931] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:08 2022] POST /identity/v3/projects => generated 359 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:08.393105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b8f8e-8781-4ebc-9340-32e015cb0e26 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.397435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3401b79-8ce5-4951-b3c3-29ecdaea1faa admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:08.462478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b8f8e-8781-4ebc-9340-32e015cb0e26 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:08.462930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:08.463803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.464080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.464347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] PATH_INFO: `/v3/projects/88805626f93d40f3b223d72c85eda4a3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.465526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] RBAC: Authorizing `identity:delete_project(project_id=88805626f93d40f3b223d72c85eda4a3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:08.467171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9e553-5027-45f9-8d61-bb5498e98212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:08.467417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:08.467865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9e553-5027-45f9-8d61-bb5498e98212 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.468177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9e553-5027-45f9-8d61-bb5498e98212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.468452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9e553-5027-45f9-8d61-bb5498e98212 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.469174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9e553-5027-45f9-8d61-bb5498e98212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:08.471113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9e553-5027-45f9-8d61-bb5498e98212 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:08.491459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 973/1932] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:08 2022] POST /identity/v3/users => generated 280 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:08.495655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9e553-5027-45f9-8d61-bb5498e98212 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.505596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9e553-5027-45f9-8d61-bb5498e98212 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:08.515114 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88805626f93d40f3b223d72c85eda4a3', 'request_id': 'req-de47248e-7fe3-4cd9-9520-5027c47b757b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.523882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88805626f93d40f3b223d72c85eda4a3', 'request_id': 'req-de47248e-7fe3-4cd9-9520-5027c47b757b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.530271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88805626f93d40f3b223d72c85eda4a3', 'request_id': 'req-de47248e-7fe3-4cd9-9520-5027c47b757b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.540573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] The token cache is being invalidate because project 88805626f93d40f3b223d72c85eda4a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:08.540861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.541566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.543142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 960/1933] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:08 2022] DELETE /identity/v3/projects/88805626f93d40f3b223d72c85eda4a3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:08.547410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.548754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:08.560901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94543a23-c1a2-4a9a-bf17-c138a0605af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:08.561662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94543a23-c1a2-4a9a-bf17-c138a0605af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.561945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94543a23-c1a2-4a9a-bf17-c138a0605af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.562224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94543a23-c1a2-4a9a-bf17-c138a0605af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.562954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94543a23-c1a2-4a9a-bf17-c138a0605af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:08.564566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94543a23-c1a2-4a9a-bf17-c138a0605af2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:08.574079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 974/1934] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:44:08.578716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94543a23-c1a2-4a9a-bf17-c138a0605af2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.580414 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-94543a23-c1a2-4a9a-bf17-c138a0605af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:08.593371 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94543a23-c1a2-4a9a-bf17-c138a0605af2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:08.598097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78c523-e898-42c0-b802-9f38f3bfb567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:08.598775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78c523-e898-42c0-b802-9f38f3bfb567 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.599059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78c523-e898-42c0-b802-9f38f3bfb567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.599335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78c523-e898-42c0-b802-9f38f3bfb567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.600208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78c523-e898-42c0-b802-9f38f3bfb567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:08.601733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78c523-e898-42c0-b802-9f38f3bfb567 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:08.613307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 975/1935] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:08.617287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78c523-e898-42c0-b802-9f38f3bfb567 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.618875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47248e-7fe3-4cd9-9520-5027c47b757b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:08.623514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:08.624478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.625538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.625721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] PATH_INFO: `/v3/users/9231880c82a847f2b23b3658bb8e1a54` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.626908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9231880c82a847f2b23b3658bb8e1a54)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:08.627731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78c523-e898-42c0-b802-9f38f3bfb567 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:08.629145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:08.633720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7884bd8-f6d7-449a-aff1-982c58eb7c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:08.634629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7884bd8-f6d7-449a-aff1-982c58eb7c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.634915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7884bd8-f6d7-449a-aff1-982c58eb7c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.635193 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7884bd8-f6d7-449a-aff1-982c58eb7c81 admin admin] PATH_INFO: `/v3/projects/192bcd6f3a374c22b7962cbe73a38fbb/users/e3eb98cd23584a6b88d87937c3ca1adc/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.650846 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7884bd8-f6d7-449a-aff1-982c58eb7c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192bcd6f3a374c22b7962cbe73a38fbb, user_id=e3eb98cd23584a6b88d87937c3ca1adc, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:08.653637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7884bd8-f6d7-449a-aff1-982c58eb7c81 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:08.667924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 976/1936] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:08 2022] PUT /identity/v3/projects/192bcd6f3a374c22b7962cbe73a38fbb/users/e3eb98cd23584a6b88d87937c3ca1adc/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:08.679398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154f7d6-553b-4013-83d5-081dd23be31e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.679691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154f7d6-553b-4013-83d5-081dd23be31e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.679972 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154f7d6-553b-4013-83d5-081dd23be31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.681285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9231880c82a847f2b23b3658bb8e1a54'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.681947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9231880c82a847f2b23b3658bb8e1a54'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.693833 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9231880c82a847f2b23b3658bb8e1a54'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.700225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9231880c82a847f2b23b3658bb8e1a54'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.706038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9154f7d6-553b-4013-83d5-081dd23be31e None None] MFA Rules not processed for user `e3eb98cd23584a6b88d87937c3ca1adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:08.707346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9154f7d6-553b-4013-83d5-081dd23be31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:08.708434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9231880c82a847f2b23b3658bb8e1a54'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:08.714805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 961/1937] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:08 2022] DELETE /identity/v3/users/9231880c82a847f2b23b3658bb8e1a54 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:08.770693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 977/1938] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:08 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:08.804041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.805558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:08.863848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556a6a6-d8e1-482b-9374-a989f761b537 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:08.867750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbabbe-a24a-4f3a-a972-96f02043bca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:08.868575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbabbe-a24a-4f3a-a972-96f02043bca2 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:08.868814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbabbe-a24a-4f3a-a972-96f02043bca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:08.869082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbabbe-a24a-4f3a-a972-96f02043bca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:08.929732 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbabbe-a24a-4f3a-a972-96f02043bca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:08.931178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbabbe-a24a-4f3a-a972-96f02043bca2 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:08.940789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 962/1939] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:08 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:08.955944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154f7d6-553b-4013-83d5-081dd23be31e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:08.958143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9154f7d6-553b-4013-83d5-081dd23be31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:09.007323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbabbe-a24a-4f3a-a972-96f02043bca2 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.008832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f2dbabbe-a24a-4f3a-a972-96f02043bca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:09.056687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154f7d6-553b-4013-83d5-081dd23be31e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.100168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbabbe-a24a-4f3a-a972-96f02043bca2 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.105671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.106775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.107255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.107664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] PATH_INFO: `/v3/projects/2ade947051be468ca93368f65b4973a6` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.109165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ade947051be468ca93368f65b4973a6)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.111478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.122304 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.123196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.123475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.123743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] PATH_INFO: `/v3/users/e3eb98cd23584a6b88d87937c3ca1adc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.125700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3eb98cd23584a6b88d87937c3ca1adc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.127717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.169969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ade947051be468ca93368f65b4973a6', 'request_id': 'req-44a20c0a-1d22-48f0-a5b1-06e10e64a103'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.178602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ade947051be468ca93368f65b4973a6', 'request_id': 'req-44a20c0a-1d22-48f0-a5b1-06e10e64a103'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.184992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3eb98cd23584a6b88d87937c3ca1adc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.185562 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ade947051be468ca93368f65b4973a6', 'request_id': 'req-44a20c0a-1d22-48f0-a5b1-06e10e64a103'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.185961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3eb98cd23584a6b88d87937c3ca1adc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.197900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] The token cache is being invalidate because project 2ade947051be468ca93368f65b4973a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:09.198288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.198998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-44a20c0a-1d22-48f0-a5b1-06e10e64a103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.200689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 963/1940] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:09 2022] DELETE /identity/v3/projects/2ade947051be468ca93368f65b4973a6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:09.201792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3eb98cd23584a6b88d87937c3ca1adc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.206404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d22c4-3a21-4f8d-8d36-63f3d01d39d1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.206693 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d22c4-3a21-4f8d-8d36-63f3d01d39d1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.206968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d22c4-3a21-4f8d-8d36-63f3d01d39d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.212299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3eb98cd23584a6b88d87937c3ca1adc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.220244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9cfce213-3bc4-4f69-ab9f-a83a57db0ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3eb98cd23584a6b88d87937c3ca1adc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.226432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 978/1941] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:08 2022] DELETE /identity/v3/users/e3eb98cd23584a6b88d87937c3ca1adc => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:09.235222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-964d22c4-3a21-4f8d-8d36-63f3d01d39d1 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:09.236151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e43a9c-abca-40bb-8fac-ac18a18a1256 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.236437 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-964d22c4-3a21-4f8d-8d36-63f3d01d39d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:09.236592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e43a9c-abca-40bb-8fac-ac18a18a1256 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.236911 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e43a9c-abca-40bb-8fac-ac18a18a1256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.253968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-18e43a9c-abca-40bb-8fac-ac18a18a1256 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:09.255199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-18e43a9c-abca-40bb-8fac-ac18a18a1256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:09.298396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 964/1942] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:09.306151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d92c99-62a6-40e7-9bb7-2bec059da212 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.306726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d92c99-62a6-40e7-9bb7-2bec059da212 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.307062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d92c99-62a6-40e7-9bb7-2bec059da212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.316912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 979/1943] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:09.325830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e3d92c99-62a6-40e7-9bb7-2bec059da212 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:09.326149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e43a9c-abca-40bb-8fac-ac18a18a1256 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.327022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e3d92c99-62a6-40e7-9bb7-2bec059da212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:09.328403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-18e43a9c-abca-40bb-8fac-ac18a18a1256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:09.335944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 965/1944] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:09.340313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d92c99-62a6-40e7-9bb7-2bec059da212 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.350687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d92c99-62a6-40e7-9bb7-2bec059da212 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.355730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708926a-ed37-45b0-bdbf-9c38779def18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.356396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708926a-ed37-45b0-bdbf-9c38779def18 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.356715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708926a-ed37-45b0-bdbf-9c38779def18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.356984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708926a-ed37-45b0-bdbf-9c38779def18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.357716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708926a-ed37-45b0-bdbf-9c38779def18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:44:09.358017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708926a-ed37-45b0-bdbf-9c38779def18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.360393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708926a-ed37-45b0-bdbf-9c38779def18 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.376733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 966/1945] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:44:09.377646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708926a-ed37-45b0-bdbf-9c38779def18 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.385393 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e43a9c-abca-40bb-8fac-ac18a18a1256 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.387237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708926a-ed37-45b0-bdbf-9c38779def18 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.389372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd1000-d327-4a82-8cc3-2e92638ff806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.390962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd1000-d327-4a82-8cc3-2e92638ff806 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.391320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd1000-d327-4a82-8cc3-2e92638ff806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.391667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd1000-d327-4a82-8cc3-2e92638ff806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.392140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23330201-0f8c-44fb-95c1-c4d5b33f7605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.392842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330201-0f8c-44fb-95c1-c4d5b33f7605 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.393125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330201-0f8c-44fb-95c1-c4d5b33f7605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.393394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330201-0f8c-44fb-95c1-c4d5b33f7605 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.394082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23330201-0f8c-44fb-95c1-c4d5b33f7605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.396041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23330201-0f8c-44fb-95c1-c4d5b33f7605 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.401680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd1000-d327-4a82-8cc3-2e92638ff806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.403109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd1000-d327-4a82-8cc3-2e92638ff806 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.414165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 980/1946] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:44:09.423981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 967/1947] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:09 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:09.428259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd1000-d327-4a82-8cc3-2e92638ff806 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.441170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd1000-d327-4a82-8cc3-2e92638ff806 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.513466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521b998-a7c0-444d-b69a-64a68e342e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.514147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521b998-a7c0-444d-b69a-64a68e342e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.514429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521b998-a7c0-444d-b69a-64a68e342e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.514783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521b998-a7c0-444d-b69a-64a68e342e9b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.515598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521b998-a7c0-444d-b69a-64a68e342e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.517601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521b998-a7c0-444d-b69a-64a68e342e9b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.543835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 981/1948] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:09 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:09.545318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23330201-0f8c-44fb-95c1-c4d5b33f7605 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.557187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23330201-0f8c-44fb-95c1-c4d5b33f7605 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.561750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64b7de-926d-4db9-b58b-c30c914b88fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.562789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64b7de-926d-4db9-b58b-c30c914b88fd admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.563073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64b7de-926d-4db9-b58b-c30c914b88fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.563944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64b7de-926d-4db9-b58b-c30c914b88fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.564784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64b7de-926d-4db9-b58b-c30c914b88fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.566395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64b7de-926d-4db9-b58b-c30c914b88fd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.576634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 968/1949] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:09.581772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521b998-a7c0-444d-b69a-64a68e342e9b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.591539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521b998-a7c0-444d-b69a-64a68e342e9b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.596233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdc33a-45d9-4950-820f-d8016db38736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.596937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc33a-45d9-4950-820f-d8016db38736 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.597215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc33a-45d9-4950-820f-d8016db38736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.597482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc33a-45d9-4950-820f-d8016db38736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.598206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdc33a-45d9-4950-820f-d8016db38736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.599803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdc33a-45d9-4950-820f-d8016db38736 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.613933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 982/1950] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:44:09.618236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64b7de-926d-4db9-b58b-c30c914b88fd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.631574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64b7de-926d-4db9-b58b-c30c914b88fd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.638648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb6b91-ace6-4706-81d7-b9ea304de029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.639842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6b91-ace6-4706-81d7-b9ea304de029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.640865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6b91-ace6-4706-81d7-b9ea304de029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.640865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6b91-ace6-4706-81d7-b9ea304de029 admin admin] PATH_INFO: `/v3/projects/e96c11f4daeb48e7af7aee6e30e58661/users/e37682766bf840309e530b3f084074ee/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.658065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb6b91-ace6-4706-81d7-b9ea304de029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96c11f4daeb48e7af7aee6e30e58661, user_id=e37682766bf840309e530b3f084074ee, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.661853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb6b91-ace6-4706-81d7-b9ea304de029 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.686899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 969/1951] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:09 2022] PUT /identity/v3/projects/e96c11f4daeb48e7af7aee6e30e58661/users/e37682766bf840309e530b3f084074ee/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:09.706158 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8f0b1-c50c-45da-acc9-66141a9551d6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.706425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8f0b1-c50c-45da-acc9-66141a9551d6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.706718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8f0b1-c50c-45da-acc9-66141a9551d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.737024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1bc8f0b1-c50c-45da-acc9-66141a9551d6 None None] MFA Rules not processed for user `e37682766bf840309e530b3f084074ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:09.738267 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1bc8f0b1-c50c-45da-acc9-66141a9551d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:09.757962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb6b91-ace6-4706-81d7-b9ea304de029 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.759690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-28eb6b91-ace6-4706-81d7-b9ea304de029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:09.816614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 983/1952] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:09.817512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8f0b1-c50c-45da-acc9-66141a9551d6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.853206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb6b91-ace6-4706-81d7-b9ea304de029 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.858050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.859745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.860029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.860302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] PATH_INFO: `/v3/projects/192bcd6f3a374c22b7962cbe73a38fbb` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.861533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] RBAC: Authorizing `identity:delete_project(project_id=192bcd6f3a374c22b7962cbe73a38fbb)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.868096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.884137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8f0b1-c50c-45da-acc9-66141a9551d6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.893445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16907896-17b6-469a-b8fe-e4f4880c6bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.894261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16907896-17b6-469a-b8fe-e4f4880c6bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.894538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16907896-17b6-469a-b8fe-e4f4880c6bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.894807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16907896-17b6-469a-b8fe-e4f4880c6bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.895518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16907896-17b6-469a-b8fe-e4f4880c6bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:09.897512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16907896-17b6-469a-b8fe-e4f4880c6bd0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:09.915259 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192bcd6f3a374c22b7962cbe73a38fbb', 'request_id': 'req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.923044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192bcd6f3a374c22b7962cbe73a38fbb', 'request_id': 'req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.924069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 984/1953] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:09 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:09.924780 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16907896-17b6-469a-b8fe-e4f4880c6bd0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.926335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-16907896-17b6-469a-b8fe-e4f4880c6bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:09.932518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192bcd6f3a374c22b7962cbe73a38fbb', 'request_id': 'req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.942909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] The token cache is being invalidate because project 192bcd6f3a374c22b7962cbe73a38fbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:09.943189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.943850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:09.945449 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 970/1954] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:09 2022] DELETE /identity/v3/projects/192bcd6f3a374c22b7962cbe73a38fbb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:09.949365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:09.950821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:09.983742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16907896-17b6-469a-b8fe-e4f4880c6bd0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:09.988022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51200bb9-13f5-444d-b239-636b74727e96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:09.988823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51200bb9-13f5-444d-b239-636b74727e96 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:09.989106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51200bb9-13f5-444d-b239-636b74727e96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:09.989381 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51200bb9-13f5-444d-b239-636b74727e96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:09.991416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-51200bb9-13f5-444d-b239-636b74727e96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:10.012391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2d0ba-20af-4e6e-8132-65ac0f8ee26f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.050143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51200bb9-13f5-444d-b239-636b74727e96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.051610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51200bb9-13f5-444d-b239-636b74727e96 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.061352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 985/1955] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:10.063767 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab77eb-2124-4717-a325-5cc3a0f57cda None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.064061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab77eb-2124-4717-a325-5cc3a0f57cda None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.064389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab77eb-2124-4717-a325-5cc3a0f57cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.085189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-12ab77eb-2124-4717-a325-5cc3a0f57cda None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:10.086669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-12ab77eb-2124-4717-a325-5cc3a0f57cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:10.095829 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e52b32-2e00-4721-b062-3fd63c7c4f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.096789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e52b32-2e00-4721-b062-3fd63c7c4f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.097213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e52b32-2e00-4721-b062-3fd63c7c4f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.097639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e52b32-2e00-4721-b062-3fd63c7c4f29 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.098341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e52b32-2e00-4721-b062-3fd63c7c4f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.100455 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e52b32-2e00-4721-b062-3fd63c7c4f29 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.126910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 971/1956] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:09 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:10.128411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e52b32-2e00-4721-b062-3fd63c7c4f29 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.138799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e52b32-2e00-4721-b062-3fd63c7c4f29 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.144038 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a67cd-8e9d-4bf8-8222-566c6de76668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.144759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a67cd-8e9d-4bf8-8222-566c6de76668 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.145764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a67cd-8e9d-4bf8-8222-566c6de76668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.146070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a67cd-8e9d-4bf8-8222-566c6de76668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.146988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a67cd-8e9d-4bf8-8222-566c6de76668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.148416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a67cd-8e9d-4bf8-8222-566c6de76668 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.150743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 986/1957] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:10.161215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b410bd3-f79f-4ba1-8465-ac74ab874011 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.161505 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b410bd3-f79f-4ba1-8465-ac74ab874011 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.161781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b410bd3-f79f-4ba1-8465-ac74ab874011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.167834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 972/1958] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:10.169754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a67cd-8e9d-4bf8-8222-566c6de76668 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.180378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a67cd-8e9d-4bf8-8222-566c6de76668 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.180867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0b410bd3-f79f-4ba1-8465-ac74ab874011 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:10.185392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0b410bd3-f79f-4ba1-8465-ac74ab874011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:10.185664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e1095-1d82-4acc-80bf-ce2daf5216eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.186837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e1095-1d82-4acc-80bf-ce2daf5216eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.187119 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e1095-1d82-4acc-80bf-ce2daf5216eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.187389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e1095-1d82-4acc-80bf-ce2daf5216eb admin admin] PATH_INFO: `/v3/projects/696427417d7049359bf65537c50374e4/users/8f27a0449586440eb9c3f238a8c6195d/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.199007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 987/1959] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:10.204402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b410bd3-f79f-4ba1-8465-ac74ab874011 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.207814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e1095-1d82-4acc-80bf-ce2daf5216eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=696427417d7049359bf65537c50374e4, user_id=8f27a0449586440eb9c3f238a8c6195d, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.211564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e1095-1d82-4acc-80bf-ce2daf5216eb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.215417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b410bd3-f79f-4ba1-8465-ac74ab874011 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.221056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eacffd-2dd1-4231-ad6a-4c41f15a1205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.221873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eacffd-2dd1-4231-ad6a-4c41f15a1205 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.222202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eacffd-2dd1-4231-ad6a-4c41f15a1205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.222413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eacffd-2dd1-4231-ad6a-4c41f15a1205 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.223150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eacffd-2dd1-4231-ad6a-4c41f15a1205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:44:10.223443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eacffd-2dd1-4231-ad6a-4c41f15a1205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.225101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eacffd-2dd1-4231-ad6a-4c41f15a1205 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.233810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 973/1960] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:10 2022] PUT /identity/v3/projects/696427417d7049359bf65537c50374e4/users/8f27a0449586440eb9c3f238a8c6195d/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:10.234975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e1095-1d82-4acc-80bf-ce2daf5216eb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.247238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 988/1961] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:44:10.248824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eacffd-2dd1-4231-ad6a-4c41f15a1205 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.261869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eacffd-2dd1-4231-ad6a-4c41f15a1205 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.347994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e1095-1d82-4acc-80bf-ce2daf5216eb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.352430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8be10a-2e0e-4600-b4da-570d48a2bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.352955 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1fd18-576d-485b-a789-22eb74c038ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.353613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8be10a-2e0e-4600-b4da-570d48a2bfb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.353720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1fd18-576d-485b-a789-22eb74c038ec admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.353975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1fd18-576d-485b-a789-22eb74c038ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.354252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1fd18-576d-485b-a789-22eb74c038ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.354716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8be10a-2e0e-4600-b4da-570d48a2bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.354967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1fd18-576d-485b-a789-22eb74c038ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.355667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8be10a-2e0e-4600-b4da-570d48a2bfb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.356552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8be10a-2e0e-4600-b4da-570d48a2bfb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.359070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8be10a-2e0e-4600-b4da-570d48a2bfb3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.360669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1fd18-576d-485b-a789-22eb74c038ec admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.379636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 974/1962] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:10.383633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1fd18-576d-485b-a789-22eb74c038ec admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.396068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1fd18-576d-485b-a789-22eb74c038ec admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.399317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 989/1963] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:10 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:10.403763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b5bbf-a31c-4f0a-88dc-5600ff87fdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.404183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8be10a-2e0e-4600-b4da-570d48a2bfb3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.404837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b5bbf-a31c-4f0a-88dc-5600ff87fdb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.405121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b5bbf-a31c-4f0a-88dc-5600ff87fdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.405715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b5bbf-a31c-4f0a-88dc-5600ff87fdb0 admin admin] PATH_INFO: `/v3/projects/696427417d7049359bf65537c50374e4/users/8f27a0449586440eb9c3f238a8c6195d/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.408315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b5bbf-a31c-4f0a-88dc-5600ff87fdb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696427417d7049359bf65537c50374e4, user_id=8f27a0449586440eb9c3f238a8c6195d, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.411148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b5bbf-a31c-4f0a-88dc-5600ff87fdb0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.424301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 975/1964] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:10 2022] PUT /identity/v3/projects/696427417d7049359bf65537c50374e4/users/8f27a0449586440eb9c3f238a8c6195d/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:10.438369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af2071-b965-4381-be07-978ecb625df5 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.438655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af2071-b965-4381-be07-978ecb625df5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.438934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af2071-b965-4381-be07-978ecb625df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.465902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-98af2071-b965-4381-be07-978ecb625df5 None None] MFA Rules not processed for user `8f27a0449586440eb9c3f238a8c6195d`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:10.467154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-98af2071-b965-4381-be07-978ecb625df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:10.480898 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8be10a-2e0e-4600-b4da-570d48a2bfb3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.536053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19471ddb-e50e-4381-a239-3f02aa763b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.536481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 976/1965] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:10.536841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19471ddb-e50e-4381-a239-3f02aa763b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.537117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19471ddb-e50e-4381-a239-3f02aa763b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.537472 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19471ddb-e50e-4381-a239-3f02aa763b29 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.538132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19471ddb-e50e-4381-a239-3f02aa763b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.540155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19471ddb-e50e-4381-a239-3f02aa763b29 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.547977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af2071-b965-4381-be07-978ecb625df5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.549476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-98af2071-b965-4381-be07-978ecb625df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:10.561128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 990/1966] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:10 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:44:10.565194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19471ddb-e50e-4381-a239-3f02aa763b29 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.576518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19471ddb-e50e-4381-a239-3f02aa763b29 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.582692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5287f76-70c8-4128-8568-3f1078a64259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.583728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5287f76-70c8-4128-8568-3f1078a64259 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.584200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5287f76-70c8-4128-8568-3f1078a64259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.584452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5287f76-70c8-4128-8568-3f1078a64259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.585229 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5287f76-70c8-4128-8568-3f1078a64259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.587194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5287f76-70c8-4128-8568-3f1078a64259 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.605346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 991/1967] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:44:10.610087 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5287f76-70c8-4128-8568-3f1078a64259 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.612248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af2071-b965-4381-be07-978ecb625df5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.617232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20326409-3513-4fe2-970d-228993b7f0fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.620069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20326409-3513-4fe2-970d-228993b7f0fb service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.620383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20326409-3513-4fe2-970d-228993b7f0fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.620700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20326409-3513-4fe2-970d-228993b7f0fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.621407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5287f76-70c8-4128-8568-3f1078a64259 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.622860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-20326409-3513-4fe2-970d-228993b7f0fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:10.628871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af639a-3805-4f34-8d35-310d8ce7a048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.629558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af639a-3805-4f34-8d35-310d8ce7a048 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.629841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af639a-3805-4f34-8d35-310d8ce7a048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.630114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af639a-3805-4f34-8d35-310d8ce7a048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.630844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af639a-3805-4f34-8d35-310d8ce7a048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.632476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af639a-3805-4f34-8d35-310d8ce7a048 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.644212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 992/1968] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:10.648455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af639a-3805-4f34-8d35-310d8ce7a048 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.659401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af639a-3805-4f34-8d35-310d8ce7a048 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.664353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d25b3b-9c70-437e-8333-ecf0e81db55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.665360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d25b3b-9c70-437e-8333-ecf0e81db55d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.665640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d25b3b-9c70-437e-8333-ecf0e81db55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.665914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d25b3b-9c70-437e-8333-ecf0e81db55d admin admin] PATH_INFO: `/v3/projects/bca9182d4e9d49e89d4e6a163c21eba5/users/94185fb7a64d4d8da8b1ce4ba30bc37f/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.681589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d25b3b-9c70-437e-8333-ecf0e81db55d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca9182d4e9d49e89d4e6a163c21eba5, user_id=94185fb7a64d4d8da8b1ce4ba30bc37f, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.684213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d25b3b-9c70-437e-8333-ecf0e81db55d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.706503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20326409-3513-4fe2-970d-228993b7f0fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.706503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20326409-3513-4fe2-970d-228993b7f0fb service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.707293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 993/1969] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:10 2022] PUT /identity/v3/projects/bca9182d4e9d49e89d4e6a163c21eba5/users/94185fb7a64d4d8da8b1ce4ba30bc37f/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:10.710237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8936b4-0b2f-44e9-8f5d-5311e86124a1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.710508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8936b4-0b2f-44e9-8f5d-5311e86124a1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.710789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8936b4-0b2f-44e9-8f5d-5311e86124a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.715261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 977/1970] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:10.739248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-aa8936b4-0b2f-44e9-8f5d-5311e86124a1 None None] MFA Rules not processed for user `94185fb7a64d4d8da8b1ce4ba30bc37f`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:10.740469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-aa8936b4-0b2f-44e9-8f5d-5311e86124a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:10.807899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 994/1971] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:10.820917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20326409-3513-4fe2-970d-228993b7f0fb service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:10.822398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-20326409-3513-4fe2-970d-228993b7f0fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:10.894265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20326409-3513-4fe2-970d-228993b7f0fb service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:10.942750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5913ce3-a445-41c0-bd25-c33d5d90b00c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6552c41f1c654ef5bfdd940e3b807dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:10.943450 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5913ce3-a445-41c0-bd25-c33d5d90b00c service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:10.943729 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5913ce3-a445-41c0-bd25-c33d5d90b00c service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:10.944005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5913ce3-a445-41c0-bd25-c33d5d90b00c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:10.952974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5913ce3-a445-41c0-bd25-c33d5d90b00c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:10.954413 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5913ce3-a445-41c0-bd25-c33d5d90b00c service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:10.964612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 978/1972] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:12.319321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8936b4-0b2f-44e9-8f5d-5311e86124a1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:12.379579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8936b4-0b2f-44e9-8f5d-5311e86124a1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:12.384179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b9248-6551-4561-aba8-46b3bd7bc6cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:12.384860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b9248-6551-4561-aba8-46b3bd7bc6cd service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:12.385133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b9248-6551-4561-aba8-46b3bd7bc6cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:12.385435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b9248-6551-4561-aba8-46b3bd7bc6cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:12.387567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-640b9248-6551-4561-aba8-46b3bd7bc6cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:12.451007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b9248-6551-4561-aba8-46b3bd7bc6cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:12.452781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b9248-6551-4561-aba8-46b3bd7bc6cd service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:12.462278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 995/1973] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:20.580015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5913ce3-a445-41c0-bd25-c33d5d90b00c service glance] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:20.589273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5913ce3-a445-41c0-bd25-c33d5d90b00c service glance] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:20.593056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a07700-6361-4046-bd72-c6895598c047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:20.593690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07700-6361-4046-bd72-c6895598c047 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:20.593966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07700-6361-4046-bd72-c6895598c047 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:20.594235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07700-6361-4046-bd72-c6895598c047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:20.657028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a07700-6361-4046-bd72-c6895598c047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:20.657866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a07700-6361-4046-bd72-c6895598c047 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:20.672663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 979/1974] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:27.129838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b9248-6551-4561-aba8-46b3bd7bc6cd service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:27.131363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-640b9248-6551-4561-aba8-46b3bd7bc6cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:27.219940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b9248-6551-4561-aba8-46b3bd7bc6cd service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:27.226336 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:27.227428 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:27.227771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:27.228110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] PATH_INFO: `/v3/users/013fd339e6954911a4568712766f61c0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:27.229496 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=013fd339e6954911a4568712766f61c0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:27.231492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:27.282913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '013fd339e6954911a4568712766f61c0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:27.283692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '013fd339e6954911a4568712766f61c0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:27.295852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '013fd339e6954911a4568712766f61c0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:27.304465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '013fd339e6954911a4568712766f61c0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:27.311792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '013fd339e6954911a4568712766f61c0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:27.318546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 996/1975] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:27 2022] DELETE /identity/v3/users/013fd339e6954911a4568712766f61c0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:27.325079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb2750-8dd6-417c-a061-17557252e2f5 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:27.325980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb2750-8dd6-417c-a061-17557252e2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:27.326527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb2750-8dd6-417c-a061-17557252e2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:27.348779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-46bb2750-8dd6-417c-a061-17557252e2f5 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:27.350364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-46bb2750-8dd6-417c-a061-17557252e2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:27.414348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 980/1976] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:27.424740 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:27.426235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:27.500807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094d493-5f3e-49ee-9306-ca72f74d38ac admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:27.504387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19276001-0dea-44a3-a77d-ce318603ac50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:27.505060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19276001-0dea-44a3-a77d-ce318603ac50 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:27.505426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19276001-0dea-44a3-a77d-ce318603ac50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:27.505766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19276001-0dea-44a3-a77d-ce318603ac50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:27.515405 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19276001-0dea-44a3-a77d-ce318603ac50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:27.516991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19276001-0dea-44a3-a77d-ce318603ac50 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:27.530773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 997/1977] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:44:27.844303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb2750-8dd6-417c-a061-17557252e2f5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:27.846003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-46bb2750-8dd6-417c-a061-17557252e2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:27.859847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb2750-8dd6-417c-a061-17557252e2f5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:27.864772 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:27.865649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:27.865924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:27.866189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] PATH_INFO: `/v3/projects/eb2b4debd8b04117bc92a18031123561` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:27.867359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb2b4debd8b04117bc92a18031123561)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:27.869382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:27.947411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb2b4debd8b04117bc92a18031123561', 'request_id': 'req-d69015f3-5a3a-4b16-8302-b1a73afed1e2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:27.959402 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb2b4debd8b04117bc92a18031123561', 'request_id': 'req-d69015f3-5a3a-4b16-8302-b1a73afed1e2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:27.967846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb2b4debd8b04117bc92a18031123561', 'request_id': 'req-d69015f3-5a3a-4b16-8302-b1a73afed1e2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:27.979547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] The token cache is being invalidate because project eb2b4debd8b04117bc92a18031123561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:27.979838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:27.980543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:27.982148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 981/1978] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:27 2022] DELETE /identity/v3/projects/eb2b4debd8b04117bc92a18031123561 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:27.986277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19276001-0dea-44a3-a77d-ce318603ac50 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:27.996372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19276001-0dea-44a3-a77d-ce318603ac50 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:28.059462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:28.060403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:28.060735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:28.061029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] PATH_INFO: `/v3/users/7625056b8b6740fd8586940a54178a2c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:28.062328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7625056b8b6740fd8586940a54178a2c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:28.064344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:28.117835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7625056b8b6740fd8586940a54178a2c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.118928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7625056b8b6740fd8586940a54178a2c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.133736 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7625056b8b6740fd8586940a54178a2c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.142974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7625056b8b6740fd8586940a54178a2c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.150947 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7625056b8b6740fd8586940a54178a2c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.158751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 998/1979] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:27 2022] DELETE /identity/v3/users/7625056b8b6740fd8586940a54178a2c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:28.565159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:28.566623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:28.583457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69015f3-5a3a-4b16-8302-b1a73afed1e2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:28.657247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:28.658140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:28.658417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:28.658689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] PATH_INFO: `/v3/projects/a1b6f1948bf14983845bfe6126bfc4e3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:28.659890 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b6f1948bf14983845bfe6126bfc4e3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:28.661937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:28.711373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b6f1948bf14983845bfe6126bfc4e3', 'request_id': 'req-275bbdd1-e455-4c0f-9a55-e4d12135af36'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.720357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b6f1948bf14983845bfe6126bfc4e3', 'request_id': 'req-275bbdd1-e455-4c0f-9a55-e4d12135af36'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.726758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b6f1948bf14983845bfe6126bfc4e3', 'request_id': 'req-275bbdd1-e455-4c0f-9a55-e4d12135af36'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.736767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] The token cache is being invalidate because project a1b6f1948bf14983845bfe6126bfc4e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:28.737074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.738222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.740234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 982/1980] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:28 2022] DELETE /identity/v3/projects/a1b6f1948bf14983845bfe6126bfc4e3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:28.744165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:28.745944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:28.759494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f63a59-727b-43c4-b386-479c9eb288d4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:28.818961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:28.819995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:28.820273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:28.820561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] PATH_INFO: `/v3/users/5c885adc357c4c669af20fe5ed4faf8d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:28.821806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c885adc357c4c669af20fe5ed4faf8d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:28.823737 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:28.873876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c885adc357c4c669af20fe5ed4faf8d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.874551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c885adc357c4c669af20fe5ed4faf8d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.886243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c885adc357c4c669af20fe5ed4faf8d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.896330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c885adc357c4c669af20fe5ed4faf8d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.903618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fe276504-5225-426a-ad11-72d09b40f972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c885adc357c4c669af20fe5ed4faf8d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:28.909600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 999/1981] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:28 2022] DELETE /identity/v3/users/5c885adc357c4c669af20fe5ed4faf8d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:29.196176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:29.197835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:29.214990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bbdd1-e455-4c0f-9a55-e4d12135af36 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:29.274749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:29.275926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:29.276288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:29.276764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] PATH_INFO: `/v3/projects/9f4978e0135a482e8c85a72cc0781e94` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:29.278799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f4978e0135a482e8c85a72cc0781e94)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:29.281785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:29.332964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f4978e0135a482e8c85a72cc0781e94', 'request_id': 'req-a1b03c05-3798-4b8f-8038-1f74897f2dbf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:29.342149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f4978e0135a482e8c85a72cc0781e94', 'request_id': 'req-a1b03c05-3798-4b8f-8038-1f74897f2dbf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:29.348725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f4978e0135a482e8c85a72cc0781e94', 'request_id': 'req-a1b03c05-3798-4b8f-8038-1f74897f2dbf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:29.359555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] The token cache is being invalidate because project 9f4978e0135a482e8c85a72cc0781e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:29.359830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:29.360518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a1b03c05-3798-4b8f-8038-1f74897f2dbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:29.362115 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 983/1982] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:29 2022] DELETE /identity/v3/projects/9f4978e0135a482e8c85a72cc0781e94 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:29.368369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff3003-b91c-4fcb-9bdf-26e2e9bc6016 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:29.368875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff3003-b91c-4fcb-9bdf-26e2e9bc6016 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:29.369028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff3003-b91c-4fcb-9bdf-26e2e9bc6016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:29.387619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-54ff3003-b91c-4fcb-9bdf-26e2e9bc6016 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:29.388851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-54ff3003-b91c-4fcb-9bdf-26e2e9bc6016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:29.453025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1000/1983] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:29.464026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a8b8e-51fa-4493-872d-7593e069d36b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:29.464227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a8b8e-51fa-4493-872d-7593e069d36b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:29.464537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a8b8e-51fa-4493-872d-7593e069d36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:29.483736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e85a8b8e-51fa-4493-872d-7593e069d36b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:29.484964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e85a8b8e-51fa-4493-872d-7593e069d36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:29.493839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 984/1984] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:29.498569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff3003-b91c-4fcb-9bdf-26e2e9bc6016 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:29.509209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff3003-b91c-4fcb-9bdf-26e2e9bc6016 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:29.513889 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafab2de-a362-495c-bc25-21e777ad33fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:29.514559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafab2de-a362-495c-bc25-21e777ad33fa admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:29.514841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafab2de-a362-495c-bc25-21e777ad33fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:29.515144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafab2de-a362-495c-bc25-21e777ad33fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:29.515877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafab2de-a362-495c-bc25-21e777ad33fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:44:29.516181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafab2de-a362-495c-bc25-21e777ad33fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:29.517838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafab2de-a362-495c-bc25-21e777ad33fa admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:29.530620 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1001/1985] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:44:29.534785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a8b8e-51fa-4493-872d-7593e069d36b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:29.544433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a8b8e-51fa-4493-872d-7593e069d36b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:29.549065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5ae4e-7a00-4136-b675-5d0693907d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:29.549850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5ae4e-7a00-4136-b675-5d0693907d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:29.550222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5ae4e-7a00-4136-b675-5d0693907d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:29.550586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5ae4e-7a00-4136-b675-5d0693907d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:29.551363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5ae4e-7a00-4136-b675-5d0693907d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:29.553650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5ae4e-7a00-4136-b675-5d0693907d8e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:29.582953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 985/1986] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:29 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:44:29.587770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafab2de-a362-495c-bc25-21e777ad33fa admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:29.598142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafab2de-a362-495c-bc25-21e777ad33fa admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:29.660517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04a941-ebd1-4753-87af-5d711708f11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:29.661243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04a941-ebd1-4753-87af-5d711708f11b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:29.661530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04a941-ebd1-4753-87af-5d711708f11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:29.661814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04a941-ebd1-4753-87af-5d711708f11b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:29.662650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04a941-ebd1-4753-87af-5d711708f11b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:29.664615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04a941-ebd1-4753-87af-5d711708f11b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:29.690946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1002/1987] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:29 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:29.692558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5ae4e-7a00-4136-b675-5d0693907d8e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:29.704083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5ae4e-7a00-4136-b675-5d0693907d8e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:29.709073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8840cd-f362-4a08-9996-ca871cafcbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:29.709747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8840cd-f362-4a08-9996-ca871cafcbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:29.710030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8840cd-f362-4a08-9996-ca871cafcbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:29.710303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8840cd-f362-4a08-9996-ca871cafcbd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:29.711033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8840cd-f362-4a08-9996-ca871cafcbd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:29.712714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8840cd-f362-4a08-9996-ca871cafcbd5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:29.724441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 986/1988] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:29.728991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04a941-ebd1-4753-87af-5d711708f11b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:29.738988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04a941-ebd1-4753-87af-5d711708f11b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:29.743999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc6ae6-808f-4836-ad6a-e8de7863b416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:29.744706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc6ae6-808f-4836-ad6a-e8de7863b416 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:29.744994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc6ae6-808f-4836-ad6a-e8de7863b416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:29.745299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc6ae6-808f-4836-ad6a-e8de7863b416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:29.746018 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc6ae6-808f-4836-ad6a-e8de7863b416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:29.747662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc6ae6-808f-4836-ad6a-e8de7863b416 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:29.757939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1003/1989] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:29.762493 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8840cd-f362-4a08-9996-ca871cafcbd5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:29.773441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8840cd-f362-4a08-9996-ca871cafcbd5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:29.779719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29faf36e-67ca-43f1-a514-9b5d6133791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:29.781137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faf36e-67ca-43f1-a514-9b5d6133791a admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:29.781137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faf36e-67ca-43f1-a514-9b5d6133791a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:29.781137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faf36e-67ca-43f1-a514-9b5d6133791a admin admin] PATH_INFO: `/v3/projects/486da17bffa0460db801eb2b986576f2/users/fa4c1acb58fb4f5ba86b1668eb4dcd03/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:29.797334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29faf36e-67ca-43f1-a514-9b5d6133791a admin admin] RBAC: Authorizing `identity:create_grant(project_id=486da17bffa0460db801eb2b986576f2, user_id=fa4c1acb58fb4f5ba86b1668eb4dcd03, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:29.800719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29faf36e-67ca-43f1-a514-9b5d6133791a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:29.817032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 987/1990] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:29 2022] PUT /identity/v3/projects/486da17bffa0460db801eb2b986576f2/users/fa4c1acb58fb4f5ba86b1668eb4dcd03/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:29.830463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c180ef9-aaee-4531-a877-dece3df0e7ad None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:29.830761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c180ef9-aaee-4531-a877-dece3df0e7ad None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:29.831037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c180ef9-aaee-4531-a877-dece3df0e7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:29.857886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0c180ef9-aaee-4531-a877-dece3df0e7ad None None] MFA Rules not processed for user `fa4c1acb58fb4f5ba86b1668eb4dcd03`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:29.859207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c180ef9-aaee-4531-a877-dece3df0e7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:29.930708 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1004/1991] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:29.946903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29faf36e-67ca-43f1-a514-9b5d6133791a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:29.949228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-29faf36e-67ca-43f1-a514-9b5d6133791a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:30.007322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29faf36e-67ca-43f1-a514-9b5d6133791a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:30.011354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ac416-963f-4ee8-aa08-a4be0f1e18a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:30.012010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ac416-963f-4ee8-aa08-a4be0f1e18a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:30.012290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ac416-963f-4ee8-aa08-a4be0f1e18a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:30.012599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ac416-963f-4ee8-aa08-a4be0f1e18a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:30.021920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ac416-963f-4ee8-aa08-a4be0f1e18a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:30.023389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ac416-963f-4ee8-aa08-a4be0f1e18a5 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:30.032963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 988/1992] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:29 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:43.208887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c180ef9-aaee-4531-a877-dece3df0e7ad None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:43.210371 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c180ef9-aaee-4531-a877-dece3df0e7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:43.277227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c180ef9-aaee-4531-a877-dece3df0e7ad None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:43.281907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:43.282796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:43.283075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:43.283344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] PATH_INFO: `/v3/users/841c42d63b5b4bf19bffabc116ddf5d8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:43.284566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=841c42d63b5b4bf19bffabc116ddf5d8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:43.286679 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:43.342170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '841c42d63b5b4bf19bffabc116ddf5d8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:43.342907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '841c42d63b5b4bf19bffabc116ddf5d8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:43.360117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '841c42d63b5b4bf19bffabc116ddf5d8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:43.373576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '841c42d63b5b4bf19bffabc116ddf5d8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:43.383026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '841c42d63b5b4bf19bffabc116ddf5d8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:43.395649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1005/1993] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:43 2022] DELETE /identity/v3/users/841c42d63b5b4bf19bffabc116ddf5d8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:43.397586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7b5d-c1d5-4b86-ade0-d44d2dfe4884 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:43.398154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7b5d-c1d5-4b86-ade0-d44d2dfe4884 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:43.398154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7b5d-c1d5-4b86-ade0-d44d2dfe4884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:43.420061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-fb3a7b5d-c1d5-4b86-ade0-d44d2dfe4884 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:43.421338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fb3a7b5d-c1d5-4b86-ade0-d44d2dfe4884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:43.512991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 989/1994] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:43.616342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:43.620547 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:43.718797 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8495933-9c03-4e0c-90f0-967d8b909308 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:43.727060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a043278-53b9-412b-bb97-d10f99462b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:43.728114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a043278-53b9-412b-bb97-d10f99462b82 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:43.728114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a043278-53b9-412b-bb97-d10f99462b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:43.728627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a043278-53b9-412b-bb97-d10f99462b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:43.740116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a043278-53b9-412b-bb97-d10f99462b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:43.742301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a043278-53b9-412b-bb97-d10f99462b82 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:43.761864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1006/1995] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:44.262755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a7b5d-c1d5-4b86-ade0-d44d2dfe4884 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:44.264273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fb3a7b5d-c1d5-4b86-ade0-d44d2dfe4884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:44.276668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a7b5d-c1d5-4b86-ade0-d44d2dfe4884 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:44.281594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:44.282540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:44.282906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:44.283244 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] PATH_INFO: `/v3/projects/ffc107ba70644002a4fd585e47c37e7f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:44.284504 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc107ba70644002a4fd585e47c37e7f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:44.286501 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:44.344324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc107ba70644002a4fd585e47c37e7f', 'request_id': 'req-2fa85e06-38b8-4106-8e09-9258a0298790'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.352522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc107ba70644002a4fd585e47c37e7f', 'request_id': 'req-2fa85e06-38b8-4106-8e09-9258a0298790'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.359760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc107ba70644002a4fd585e47c37e7f', 'request_id': 'req-2fa85e06-38b8-4106-8e09-9258a0298790'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.371878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] The token cache is being invalidate because project ffc107ba70644002a4fd585e47c37e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:44.372188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.372908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.374481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 990/1996] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:44 2022] DELETE /identity/v3/projects/ffc107ba70644002a4fd585e47c37e7f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:44.378896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a043278-53b9-412b-bb97-d10f99462b82 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:44.391688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a043278-53b9-412b-bb97-d10f99462b82 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:44.450970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:44.452373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:44.452876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:44.453215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] PATH_INFO: `/v3/users/1aafa0f82dd44b84a8239717fd9a411e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:44.454725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aafa0f82dd44b84a8239717fd9a411e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:44.457765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:44.521015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aafa0f82dd44b84a8239717fd9a411e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.521681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aafa0f82dd44b84a8239717fd9a411e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.534630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aafa0f82dd44b84a8239717fd9a411e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.542157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aafa0f82dd44b84a8239717fd9a411e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.549389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9df78619-fc48-478a-9041-3cdb5ad97584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aafa0f82dd44b84a8239717fd9a411e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.555330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1007/1997] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:44 2022] DELETE /identity/v3/users/1aafa0f82dd44b84a8239717fd9a411e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:44.840944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:44.842900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:44.858670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa85e06-38b8-4106-8e09-9258a0298790 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:44.918261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:44.919246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:44.919615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:44.919991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] PATH_INFO: `/v3/projects/7284f076fb334864b4aad41d2640b239` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:44.921328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7284f076fb334864b4aad41d2640b239)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:44.923419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:44.985911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7284f076fb334864b4aad41d2640b239', 'request_id': 'req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:44.997830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7284f076fb334864b4aad41d2640b239', 'request_id': 'req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:45.006570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7284f076fb334864b4aad41d2640b239', 'request_id': 'req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:45.022570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] The token cache is being invalidate because project 7284f076fb334864b4aad41d2640b239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:45.023060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:45.023793 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6de52b3a-0b89-4a1a-ba9d-29f3f33ffb3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:45.025644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 991/1998] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:44 2022] DELETE /identity/v3/projects/7284f076fb334864b4aad41d2640b239 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:45.030833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f824b1-cc0c-43ff-bcdc-68cc66b8d640 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.031119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f824b1-cc0c-43ff-bcdc-68cc66b8d640 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.031569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f824b1-cc0c-43ff-bcdc-68cc66b8d640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.054438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-06f824b1-cc0c-43ff-bcdc-68cc66b8d640 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:45.057218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-06f824b1-cc0c-43ff-bcdc-68cc66b8d640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:45.124117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1008/1999] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:45.131941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7488cc-56a5-4055-8fca-bd8fc6e1627a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.132311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7488cc-56a5-4055-8fca-bd8fc6e1627a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.132702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7488cc-56a5-4055-8fca-bd8fc6e1627a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.153529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ac7488cc-56a5-4055-8fca-bd8fc6e1627a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:45.154843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ac7488cc-56a5-4055-8fca-bd8fc6e1627a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:45.163260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 992/2000] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:45.167747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f824b1-cc0c-43ff-bcdc-68cc66b8d640 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.177931 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f824b1-cc0c-43ff-bcdc-68cc66b8d640 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.185326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80509ff5-68b8-4468-a9c4-e77aac71af08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.186002 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80509ff5-68b8-4468-a9c4-e77aac71af08 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.186286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80509ff5-68b8-4468-a9c4-e77aac71af08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.186577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80509ff5-68b8-4468-a9c4-e77aac71af08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.187282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80509ff5-68b8-4468-a9c4-e77aac71af08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:44:45.187584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80509ff5-68b8-4468-a9c4-e77aac71af08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.189315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80509ff5-68b8-4468-a9c4-e77aac71af08 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.216532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1009/2001] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:44:45.220671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7488cc-56a5-4055-8fca-bd8fc6e1627a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.231213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7488cc-56a5-4055-8fca-bd8fc6e1627a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.235134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a9b18-8ded-4494-ba13-6bd0f6ec5d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.235807 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a9b18-8ded-4494-ba13-6bd0f6ec5d86 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.236085 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a9b18-8ded-4494-ba13-6bd0f6ec5d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.236355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a9b18-8ded-4494-ba13-6bd0f6ec5d86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.237194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a9b18-8ded-4494-ba13-6bd0f6ec5d86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.239501 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a9b18-8ded-4494-ba13-6bd0f6ec5d86 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.269138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 993/2002] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:45 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:45.273375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80509ff5-68b8-4468-a9c4-e77aac71af08 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.286911 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80509ff5-68b8-4468-a9c4-e77aac71af08 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.353236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6169ab8-066f-4a2f-bf38-e332bbf61bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.354053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6169ab8-066f-4a2f-bf38-e332bbf61bab admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.354666 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6169ab8-066f-4a2f-bf38-e332bbf61bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.354963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6169ab8-066f-4a2f-bf38-e332bbf61bab admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.355834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6169ab8-066f-4a2f-bf38-e332bbf61bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.357796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6169ab8-066f-4a2f-bf38-e332bbf61bab admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.378801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1010/2003] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:45 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:45.383071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a9b18-8ded-4494-ba13-6bd0f6ec5d86 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.393515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a9b18-8ded-4494-ba13-6bd0f6ec5d86 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.398439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9e4d8-bb13-4cca-8ff3-93152063e430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.399102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e4d8-bb13-4cca-8ff3-93152063e430 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.399399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e4d8-bb13-4cca-8ff3-93152063e430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.399665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e4d8-bb13-4cca-8ff3-93152063e430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.400366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9e4d8-bb13-4cca-8ff3-93152063e430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.402047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9e4d8-bb13-4cca-8ff3-93152063e430 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.412258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 994/2004] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:45.416305 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6169ab8-066f-4a2f-bf38-e332bbf61bab admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.426084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6169ab8-066f-4a2f-bf38-e332bbf61bab admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.432041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133fdb1-3ae1-49d7-8642-bb192d52a4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.432744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133fdb1-3ae1-49d7-8642-bb192d52a4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.433330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133fdb1-3ae1-49d7-8642-bb192d52a4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.433606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133fdb1-3ae1-49d7-8642-bb192d52a4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.434344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133fdb1-3ae1-49d7-8642-bb192d52a4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.435942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133fdb1-3ae1-49d7-8642-bb192d52a4bf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.446882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1011/2005] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:45.452158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9e4d8-bb13-4cca-8ff3-93152063e430 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.462421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9e4d8-bb13-4cca-8ff3-93152063e430 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.467586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089fca3-bb03-4c6f-8c25-520b9e28661a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.468518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089fca3-bb03-4c6f-8c25-520b9e28661a admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.468811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089fca3-bb03-4c6f-8c25-520b9e28661a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.469087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089fca3-bb03-4c6f-8c25-520b9e28661a admin admin] PATH_INFO: `/v3/projects/034148892c2f4780880bd1fcda64b48f/users/112ce7527da44264bbf35aa393758e5c/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.484688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089fca3-bb03-4c6f-8c25-520b9e28661a admin admin] RBAC: Authorizing `identity:create_grant(project_id=034148892c2f4780880bd1fcda64b48f, user_id=112ce7527da44264bbf35aa393758e5c, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.487345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089fca3-bb03-4c6f-8c25-520b9e28661a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.500988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 995/2006] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:45 2022] PUT /identity/v3/projects/034148892c2f4780880bd1fcda64b48f/users/112ce7527da44264bbf35aa393758e5c/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:45.511711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c20a1-b1a0-4993-a49d-96368517f112 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.511937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c20a1-b1a0-4993-a49d-96368517f112 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.512217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c20a1-b1a0-4993-a49d-96368517f112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.538484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-666c20a1-b1a0-4993-a49d-96368517f112 None None] MFA Rules not processed for user `112ce7527da44264bbf35aa393758e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:45.539665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-666c20a1-b1a0-4993-a49d-96368517f112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:45.604885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1012/2007] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:45.609481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089fca3-bb03-4c6f-8c25-520b9e28661a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.672225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089fca3-bb03-4c6f-8c25-520b9e28661a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.676770 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20ae95-fa43-434e-8324-ee932bbb16bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.677433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20ae95-fa43-434e-8324-ee932bbb16bc admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.677710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20ae95-fa43-434e-8324-ee932bbb16bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.677980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20ae95-fa43-434e-8324-ee932bbb16bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.678663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20ae95-fa43-434e-8324-ee932bbb16bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.680609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20ae95-fa43-434e-8324-ee932bbb16bc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.711254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 996/2008] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:45 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:45.713219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c20a1-b1a0-4993-a49d-96368517f112 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.724756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c20a1-b1a0-4993-a49d-96368517f112 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.785243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27598ce8-2626-4ed3-8c12-bae6156281b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.786641 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598ce8-2626-4ed3-8c12-bae6156281b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.787037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598ce8-2626-4ed3-8c12-bae6156281b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.787366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598ce8-2626-4ed3-8c12-bae6156281b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.788137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27598ce8-2626-4ed3-8c12-bae6156281b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.790747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27598ce8-2626-4ed3-8c12-bae6156281b2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.815578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1013/2009] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:45 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:45.819515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20ae95-fa43-434e-8324-ee932bbb16bc admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.830349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20ae95-fa43-434e-8324-ee932bbb16bc admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.835780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7087098-7e20-48d4-84c5-6e28b20c3ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.836463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7087098-7e20-48d4-84c5-6e28b20c3ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.836789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7087098-7e20-48d4-84c5-6e28b20c3ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.837047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7087098-7e20-48d4-84c5-6e28b20c3ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.837798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7087098-7e20-48d4-84c5-6e28b20c3ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.839446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7087098-7e20-48d4-84c5-6e28b20c3ac6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.851737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 997/2010] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:45.856170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27598ce8-2626-4ed3-8c12-bae6156281b2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.866279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27598ce8-2626-4ed3-8c12-bae6156281b2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.872922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f534-b906-4bb0-900e-766c750dd2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.873902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f534-b906-4bb0-900e-766c750dd2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.874248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f534-b906-4bb0-900e-766c750dd2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.874609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f534-b906-4bb0-900e-766c750dd2f1 admin admin] PATH_INFO: `/v3/projects/ed1b7b51f31641969c5a501470108526/users/0321f81981484594b7da413c164985a1/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.893378 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f534-b906-4bb0-900e-766c750dd2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1b7b51f31641969c5a501470108526, user_id=0321f81981484594b7da413c164985a1, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.896223 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f534-b906-4bb0-900e-766c750dd2f1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.911293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1014/2011] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:45 2022] PUT /identity/v3/projects/ed1b7b51f31641969c5a501470108526/users/0321f81981484594b7da413c164985a1/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:45.915260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7087098-7e20-48d4-84c5-6e28b20c3ac6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:45.925288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7087098-7e20-48d4-84c5-6e28b20c3ac6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:45.979041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba628b-6960-48e2-a0c1-58bd9faec172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:45.979712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba628b-6960-48e2-a0c1-58bd9faec172 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:45.980030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba628b-6960-48e2-a0c1-58bd9faec172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:45.980307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba628b-6960-48e2-a0c1-58bd9faec172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:45.981089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba628b-6960-48e2-a0c1-58bd9faec172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:45.982684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba628b-6960-48e2-a0c1-58bd9faec172 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:45.992149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 998/2012] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:44:45.996772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f534-b906-4bb0-900e-766c750dd2f1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:46.007262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f534-b906-4bb0-900e-766c750dd2f1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:46.013759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290f771-ba38-4158-a152-d417f061a7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:46.014679 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290f771-ba38-4158-a152-d417f061a7b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:46.014959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290f771-ba38-4158-a152-d417f061a7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:46.015230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290f771-ba38-4158-a152-d417f061a7b2 admin admin] PATH_INFO: `/v3/projects/ed1b7b51f31641969c5a501470108526/users/0321f81981484594b7da413c164985a1/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:46.017444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290f771-ba38-4158-a152-d417f061a7b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1b7b51f31641969c5a501470108526, user_id=0321f81981484594b7da413c164985a1, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:46.020118 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290f771-ba38-4158-a152-d417f061a7b2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:46.031993 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1015/2013] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:45 2022] PUT /identity/v3/projects/ed1b7b51f31641969c5a501470108526/users/0321f81981484594b7da413c164985a1/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:46.043620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1fde2-9883-4a28-9bd5-5d30da8e8822 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:46.043914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1fde2-9883-4a28-9bd5-5d30da8e8822 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:46.044197 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1fde2-9883-4a28-9bd5-5d30da8e8822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:46.077919 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e4f1fde2-9883-4a28-9bd5-5d30da8e8822 None None] MFA Rules not processed for user `0321f81981484594b7da413c164985a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:46.079945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e4f1fde2-9883-4a28-9bd5-5d30da8e8822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:46.155100 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 999/2014] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:46.166393 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290f771-ba38-4158-a152-d417f061a7b2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:46.168345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2290f771-ba38-4158-a152-d417f061a7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:46.225115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290f771-ba38-4158-a152-d417f061a7b2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:46.228748 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab52ae-78d8-49fa-a636-cbea51749756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:46.229388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab52ae-78d8-49fa-a636-cbea51749756 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:46.229667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab52ae-78d8-49fa-a636-cbea51749756 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:46.229959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab52ae-78d8-49fa-a636-cbea51749756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:46.241155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab52ae-78d8-49fa-a636-cbea51749756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:46.242583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab52ae-78d8-49fa-a636-cbea51749756 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:46.254069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1016/2015] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:46.284342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1fde2-9883-4a28-9bd5-5d30da8e8822 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:46.347741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1fde2-9883-4a28-9bd5-5d30da8e8822 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:46.352594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:46.353763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:46.354028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:46.354206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] PATH_INFO: `/v3/users/112ce7527da44264bbf35aa393758e5c` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:46.360997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=112ce7527da44264bbf35aa393758e5c)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:46.363199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:46.411695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112ce7527da44264bbf35aa393758e5c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:46.412338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112ce7527da44264bbf35aa393758e5c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:46.423839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112ce7527da44264bbf35aa393758e5c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:46.430311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112ce7527da44264bbf35aa393758e5c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:46.439015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112ce7527da44264bbf35aa393758e5c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:46.445641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1000/2016] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:46 2022] DELETE /identity/v3/users/112ce7527da44264bbf35aa393758e5c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:46.450344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce082b-72fd-4a8a-b203-2b2e0e364a0a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:46.450824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce082b-72fd-4a8a-b203-2b2e0e364a0a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:46.450915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce082b-72fd-4a8a-b203-2b2e0e364a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:46.469777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-99ce082b-72fd-4a8a-b203-2b2e0e364a0a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:46.471230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-99ce082b-72fd-4a8a-b203-2b2e0e364a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:46.528579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1017/2017] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:46.538291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:46.539602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:46.600806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190961eb-f26d-46a2-9ede-3563494c9d30 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:46.605007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b396c-899d-4131-92af-c89b8e536fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:46.605800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b396c-899d-4131-92af-c89b8e536fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:46.606182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b396c-899d-4131-92af-c89b8e536fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:46.606453 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b396c-899d-4131-92af-c89b8e536fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:46.617044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b396c-899d-4131-92af-c89b8e536fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:46.618707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b396c-899d-4131-92af-c89b8e536fc9 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:46.629361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1001/2018] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:46.914777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce082b-72fd-4a8a-b203-2b2e0e364a0a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:46.916427 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-99ce082b-72fd-4a8a-b203-2b2e0e364a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:46.929323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce082b-72fd-4a8a-b203-2b2e0e364a0a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:46.936257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:46.937173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:46.937451 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:46.937719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] PATH_INFO: `/v3/projects/034148892c2f4780880bd1fcda64b48f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:46.938935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=034148892c2f4780880bd1fcda64b48f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:46.941724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:46.989861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '034148892c2f4780880bd1fcda64b48f', 'request_id': 'req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:46.997694 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '034148892c2f4780880bd1fcda64b48f', 'request_id': 'req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.003710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '034148892c2f4780880bd1fcda64b48f', 'request_id': 'req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.013538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] The token cache is being invalidate because project 034148892c2f4780880bd1fcda64b48f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:47.013820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.014438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.015981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1018/2019] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:46 2022] DELETE /identity/v3/projects/034148892c2f4780880bd1fcda64b48f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:47.020102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b396c-899d-4131-92af-c89b8e536fc9 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:47.030009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b396c-899d-4131-92af-c89b8e536fc9 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:47.087083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:47.087884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:47.088165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:47.088436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] PATH_INFO: `/v3/users/0321f81981484594b7da413c164985a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:47.089763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0321f81981484594b7da413c164985a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:47.091690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:47.142613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0321f81981484594b7da413c164985a1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.143264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0321f81981484594b7da413c164985a1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.155091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0321f81981484594b7da413c164985a1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.161628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0321f81981484594b7da413c164985a1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.169632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8e0f381c-2323-48d9-a65d-4ab704727d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0321f81981484594b7da413c164985a1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.176024 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1002/2020] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:44:47 2022] DELETE /identity/v3/users/0321f81981484594b7da413c164985a1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:47.456413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:47.458523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:47.474214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9da09e-f959-41c9-b434-6ecc1d4e3a49 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:47.530293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:47.531180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:47.531452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:47.531723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] PATH_INFO: `/v3/projects/ed1b7b51f31641969c5a501470108526` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:47.533012 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1b7b51f31641969c5a501470108526)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:47.534894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:47.582443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1b7b51f31641969c5a501470108526', 'request_id': 'req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.591320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1b7b51f31641969c5a501470108526', 'request_id': 'req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.597863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1b7b51f31641969c5a501470108526', 'request_id': 'req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.612679 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] The token cache is being invalidate because project ed1b7b51f31641969c5a501470108526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:44:47.612969 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.613628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12f07d8a-535f-4d89-8fbd-8fbf8d48ceb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:44:47.615478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1019/2021] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:44:47 2022] DELETE /identity/v3/projects/ed1b7b51f31641969c5a501470108526 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:47.620412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19bc11-a28a-4b64-84eb-92d74b959e44 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:47.621727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19bc11-a28a-4b64-84eb-92d74b959e44 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:47.622109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19bc11-a28a-4b64-84eb-92d74b959e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:47.643852 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-da19bc11-a28a-4b64-84eb-92d74b959e44 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:47.645093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-da19bc11-a28a-4b64-84eb-92d74b959e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:47.704692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1003/2022] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:47.712455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da4e17-733c-4fce-8293-5c69a3644d85 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:47.712791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da4e17-733c-4fce-8293-5c69a3644d85 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:47.713067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da4e17-733c-4fce-8293-5c69a3644d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:47.733407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b6da4e17-733c-4fce-8293-5c69a3644d85 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:47.734984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b6da4e17-733c-4fce-8293-5c69a3644d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:47.747568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1020/2023] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:47.752071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19bc11-a28a-4b64-84eb-92d74b959e44 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:47.762290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19bc11-a28a-4b64-84eb-92d74b959e44 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:47.767807 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38854d01-84fe-457e-b3e9-5fb16e07c1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:47.768526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854d01-84fe-457e-b3e9-5fb16e07c1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:47.768848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854d01-84fe-457e-b3e9-5fb16e07c1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:47.769127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854d01-84fe-457e-b3e9-5fb16e07c1b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:47.769867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38854d01-84fe-457e-b3e9-5fb16e07c1b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:44:47.770167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38854d01-84fe-457e-b3e9-5fb16e07c1b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:47.771791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38854d01-84fe-457e-b3e9-5fb16e07c1b9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:47.783623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1004/2024] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:44:47.787932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da4e17-733c-4fce-8293-5c69a3644d85 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:47.797596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da4e17-733c-4fce-8293-5c69a3644d85 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:47.803225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a537c79-1769-405d-b9d4-3d316dc80452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:47.803879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a537c79-1769-405d-b9d4-3d316dc80452 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:47.804156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a537c79-1769-405d-b9d4-3d316dc80452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:47.804425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a537c79-1769-405d-b9d4-3d316dc80452 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:47.805153 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a537c79-1769-405d-b9d4-3d316dc80452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:47.807040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a537c79-1769-405d-b9d4-3d316dc80452 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:47.836658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1021/2025] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:47 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:47.840017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38854d01-84fe-457e-b3e9-5fb16e07c1b9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:47.850269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38854d01-84fe-457e-b3e9-5fb16e07c1b9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:47.916247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db994c8c-321b-4ab8-8bb8-ea89cd51e2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:47.916941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db994c8c-321b-4ab8-8bb8-ea89cd51e2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:47.917227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db994c8c-321b-4ab8-8bb8-ea89cd51e2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:47.917560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db994c8c-321b-4ab8-8bb8-ea89cd51e2d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:47.918258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db994c8c-321b-4ab8-8bb8-ea89cd51e2d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:47.920171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db994c8c-321b-4ab8-8bb8-ea89cd51e2d7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:47.944601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1005/2026] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:47 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:44:47.949162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a537c79-1769-405d-b9d4-3d316dc80452 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:47.960077 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a537c79-1769-405d-b9d4-3d316dc80452 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:47.965102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dabb59-f705-449c-a9f7-3bdb79e94602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:47.965751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dabb59-f705-449c-a9f7-3bdb79e94602 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:47.966034 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dabb59-f705-449c-a9f7-3bdb79e94602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:47.966311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dabb59-f705-449c-a9f7-3bdb79e94602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:47.967072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dabb59-f705-449c-a9f7-3bdb79e94602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:47.968696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dabb59-f705-449c-a9f7-3bdb79e94602 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:47.979201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1022/2027] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:47.983438 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db994c8c-321b-4ab8-8bb8-ea89cd51e2d7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:47.993278 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db994c8c-321b-4ab8-8bb8-ea89cd51e2d7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:47.998476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0082ef-9c59-42ac-85ec-92f04b5ad2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:47.999148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0082ef-9c59-42ac-85ec-92f04b5ad2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:47.999457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0082ef-9c59-42ac-85ec-92f04b5ad2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:47.999733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0082ef-9c59-42ac-85ec-92f04b5ad2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.000460 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0082ef-9c59-42ac-85ec-92f04b5ad2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:48.002176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0082ef-9c59-42ac-85ec-92f04b5ad2a6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:48.012613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1006/2028] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:48.016806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dabb59-f705-449c-a9f7-3bdb79e94602 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:48.026619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dabb59-f705-449c-a9f7-3bdb79e94602 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:48.031310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf3d15-ec1a-410c-b910-9f72c9c6ded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:48.032201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf3d15-ec1a-410c-b910-9f72c9c6ded9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.032476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf3d15-ec1a-410c-b910-9f72c9c6ded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.032776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf3d15-ec1a-410c-b910-9f72c9c6ded9 admin admin] PATH_INFO: `/v3/projects/6a415ebd578949ad807baf0947dd82ec/users/00530d819ab947b9b4ef5a981cd060b8/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.047057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf3d15-ec1a-410c-b910-9f72c9c6ded9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a415ebd578949ad807baf0947dd82ec, user_id=00530d819ab947b9b4ef5a981cd060b8, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:48.049795 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf3d15-ec1a-410c-b910-9f72c9c6ded9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:48.062873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1023/2029] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:48 2022] PUT /identity/v3/projects/6a415ebd578949ad807baf0947dd82ec/users/00530d819ab947b9b4ef5a981cd060b8/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:48.074532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fcd15-95c3-4477-a70b-89cd1cf4623a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.075073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fcd15-95c3-4477-a70b-89cd1cf4623a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.075073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fcd15-95c3-4477-a70b-89cd1cf4623a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.101318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-856fcd15-95c3-4477-a70b-89cd1cf4623a None None] MFA Rules not processed for user `00530d819ab947b9b4ef5a981cd060b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:48.102387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-856fcd15-95c3-4477-a70b-89cd1cf4623a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:48.169045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1007/2030] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:48 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:44:48.173756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf3d15-ec1a-410c-b910-9f72c9c6ded9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:48.234723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf3d15-ec1a-410c-b910-9f72c9c6ded9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:48.239858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663420-1131-450a-8d26-3ebd890fc420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:48.240526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663420-1131-450a-8d26-3ebd890fc420 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.240837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663420-1131-450a-8d26-3ebd890fc420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.241102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663420-1131-450a-8d26-3ebd890fc420 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.241795 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13663420-1131-450a-8d26-3ebd890fc420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:48.243708 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13663420-1131-450a-8d26-3ebd890fc420 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:48.270269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1024/2031] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:44:48 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:48.274593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fcd15-95c3-4477-a70b-89cd1cf4623a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:48.285496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fcd15-95c3-4477-a70b-89cd1cf4623a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:48.349238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb51c14-b506-4645-a0cc-a0462f1a90d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:48.349979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb51c14-b506-4645-a0cc-a0462f1a90d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.350270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb51c14-b506-4645-a0cc-a0462f1a90d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.350544 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb51c14-b506-4645-a0cc-a0462f1a90d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.351234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb51c14-b506-4645-a0cc-a0462f1a90d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:48.353197 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb51c14-b506-4645-a0cc-a0462f1a90d5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:48.375447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1008/2032] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:44:48 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:44:48.379618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663420-1131-450a-8d26-3ebd890fc420 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:48.390105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663420-1131-450a-8d26-3ebd890fc420 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:48.395367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12c9cf-7770-483c-86cd-7cd67035bab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:48.396067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12c9cf-7770-483c-86cd-7cd67035bab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.396346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12c9cf-7770-483c-86cd-7cd67035bab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.396644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12c9cf-7770-483c-86cd-7cd67035bab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.397364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12c9cf-7770-483c-86cd-7cd67035bab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:48.398976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12c9cf-7770-483c-86cd-7cd67035bab1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:48.410217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1025/2033] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:48.414415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb51c14-b506-4645-a0cc-a0462f1a90d5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:48.424999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb51c14-b506-4645-a0cc-a0462f1a90d5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:48.430209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f906f9-0305-41e4-a6ca-e8ee930f8bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:48.431277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f906f9-0305-41e4-a6ca-e8ee930f8bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.431654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f906f9-0305-41e4-a6ca-e8ee930f8bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.431914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f906f9-0305-41e4-a6ca-e8ee930f8bd9 admin admin] PATH_INFO: `/v3/projects/64bbc33eaf0f4e0299df24427f42653b/users/607e48841d7144feb0d10dce709e0e71/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.459899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f906f9-0305-41e4-a6ca-e8ee930f8bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64bbc33eaf0f4e0299df24427f42653b, user_id=607e48841d7144feb0d10dce709e0e71, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:48.463097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f906f9-0305-41e4-a6ca-e8ee930f8bd9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:48.477437 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1009/2034] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:48 2022] PUT /identity/v3/projects/64bbc33eaf0f4e0299df24427f42653b/users/607e48841d7144feb0d10dce709e0e71/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:44:48.481805 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12c9cf-7770-483c-86cd-7cd67035bab1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:48.492103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12c9cf-7770-483c-86cd-7cd67035bab1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:48.547055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de9aab-b75e-4891-8153-90559fe4c17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:48.547727 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de9aab-b75e-4891-8153-90559fe4c17b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.548070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de9aab-b75e-4891-8153-90559fe4c17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.548329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de9aab-b75e-4891-8153-90559fe4c17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.549082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de9aab-b75e-4891-8153-90559fe4c17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:48.550675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de9aab-b75e-4891-8153-90559fe4c17b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:48.561440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1026/2035] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:44:48.565672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f906f9-0305-41e4-a6ca-e8ee930f8bd9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:48.575979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f906f9-0305-41e4-a6ca-e8ee930f8bd9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:48.581171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128aa6c9-f43b-4f3f-8519-6cddf7956701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:48.582068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aa6c9-f43b-4f3f-8519-6cddf7956701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.582352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aa6c9-f43b-4f3f-8519-6cddf7956701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.582626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aa6c9-f43b-4f3f-8519-6cddf7956701 admin admin] PATH_INFO: `/v3/projects/64bbc33eaf0f4e0299df24427f42653b/users/607e48841d7144feb0d10dce709e0e71/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.585257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128aa6c9-f43b-4f3f-8519-6cddf7956701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64bbc33eaf0f4e0299df24427f42653b, user_id=607e48841d7144feb0d10dce709e0e71, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:48.587484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128aa6c9-f43b-4f3f-8519-6cddf7956701 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:48.600803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1010/2036] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:44:48 2022] PUT /identity/v3/projects/64bbc33eaf0f4e0299df24427f42653b/users/607e48841d7144feb0d10dce709e0e71/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:44:48.612304 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7ed6a-b35e-4018-8f0f-d38c2602be54 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.612612 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7ed6a-b35e-4018-8f0f-d38c2602be54 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.612960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7ed6a-b35e-4018-8f0f-d38c2602be54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.639705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-acd7ed6a-b35e-4018-8f0f-d38c2602be54 None None] MFA Rules not processed for user `607e48841d7144feb0d10dce709e0e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:44:48.640948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-acd7ed6a-b35e-4018-8f0f-d38c2602be54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:48.708321 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1027/2037] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:44:48 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:44:48.720304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128aa6c9-f43b-4f3f-8519-6cddf7956701 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:48.722143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-128aa6c9-f43b-4f3f-8519-6cddf7956701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:44:48.780906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128aa6c9-f43b-4f3f-8519-6cddf7956701 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:48.785105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772f421-2195-4c76-88da-495139088b63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:48.785793 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772f421-2195-4c76-88da-495139088b63 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.786074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772f421-2195-4c76-88da-495139088b63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.786368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772f421-2195-4c76-88da-495139088b63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.795960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772f421-2195-4c76-88da-495139088b63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:48.797432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772f421-2195-4c76-88da-495139088b63 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:48.808194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1011/2038] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:44:48.893911 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7ed6a-b35e-4018-8f0f-d38c2602be54 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:44:48.903916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7ed6a-b35e-4018-8f0f-d38c2602be54 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:44:48.908337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250d980-599d-40cd-9d22-944d29986203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:44:48.909053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250d980-599d-40cd-9d22-944d29986203 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:44:48.909337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250d980-599d-40cd-9d22-944d29986203 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:44:48.909632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250d980-599d-40cd-9d22-944d29986203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:44:48.969231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250d980-599d-40cd-9d22-944d29986203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:44:48.970674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250d980-599d-40cd-9d22-944d29986203 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:44:48.979692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1028/2039] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:44:48 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:02.254406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772f421-2195-4c76-88da-495139088b63 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:02.317823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772f421-2195-4c76-88da-495139088b63 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:02.323735 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:02.324650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:02.324934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:02.325205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] PATH_INFO: `/v3/users/00530d819ab947b9b4ef5a981cd060b8` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:02.326821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00530d819ab947b9b4ef5a981cd060b8)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:02.328926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:02.383718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00530d819ab947b9b4ef5a981cd060b8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:02.384625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00530d819ab947b9b4ef5a981cd060b8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:02.398399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00530d819ab947b9b4ef5a981cd060b8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:02.405513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00530d819ab947b9b4ef5a981cd060b8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:02.414674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00530d819ab947b9b4ef5a981cd060b8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:02.421095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1012/2040] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:02 2022] DELETE /identity/v3/users/00530d819ab947b9b4ef5a981cd060b8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:02.425821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d66e2-7c81-4fbd-8a31-9630d6f8d3e9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:02.426330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d66e2-7c81-4fbd-8a31-9630d6f8d3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:02.426439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d66e2-7c81-4fbd-8a31-9630d6f8d3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:02.443560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ad4d66e2-7c81-4fbd-8a31-9630d6f8d3e9 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:02.444859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ad4d66e2-7c81-4fbd-8a31-9630d6f8d3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:02.507105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1029/2041] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:02.518130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:02.520110 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:02.578358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb0fbd-39ab-42b8-b18d-ce7cbaec4353 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:02.582974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c3ea8-5827-4f91-abdb-9c371f24abaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:02.583662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c3ea8-5827-4f91-abdb-9c371f24abaf service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:02.583947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c3ea8-5827-4f91-abdb-9c371f24abaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:02.584216 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c3ea8-5827-4f91-abdb-9c371f24abaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:02.594459 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c3ea8-5827-4f91-abdb-9c371f24abaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:02.595879 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c3ea8-5827-4f91-abdb-9c371f24abaf service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:02.606005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1013/2042] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:02.873678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d66e2-7c81-4fbd-8a31-9630d6f8d3e9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:02.875402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ad4d66e2-7c81-4fbd-8a31-9630d6f8d3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:02.892748 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d66e2-7c81-4fbd-8a31-9630d6f8d3e9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:02.897951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:02.898802 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:02.899074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:02.899322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] PATH_INFO: `/v3/projects/6a415ebd578949ad807baf0947dd82ec` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:02.902072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a415ebd578949ad807baf0947dd82ec)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:02.904000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:02.959190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a415ebd578949ad807baf0947dd82ec', 'request_id': 'req-1e07c314-b7ea-430a-9c34-6a6f38cd6679'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:02.969848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a415ebd578949ad807baf0947dd82ec', 'request_id': 'req-1e07c314-b7ea-430a-9c34-6a6f38cd6679'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:02.975189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a415ebd578949ad807baf0947dd82ec', 'request_id': 'req-1e07c314-b7ea-430a-9c34-6a6f38cd6679'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:02.986007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] The token cache is being invalidate because project 6a415ebd578949ad807baf0947dd82ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:02.986401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:02.987420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:02.998999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1030/2043] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:02 2022] DELETE /identity/v3/projects/6a415ebd578949ad807baf0947dd82ec => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:02.999785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c3ea8-5827-4f91-abdb-9c371f24abaf service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:03.004692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c3ea8-5827-4f91-abdb-9c371f24abaf service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:03.083185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:03.084222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:03.084603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:03.084949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] PATH_INFO: `/v3/users/607e48841d7144feb0d10dce709e0e71` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:03.086309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=607e48841d7144feb0d10dce709e0e71)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:03.088688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:03.149868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607e48841d7144feb0d10dce709e0e71'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:03.150594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607e48841d7144feb0d10dce709e0e71'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:03.164667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607e48841d7144feb0d10dce709e0e71'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:03.171008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607e48841d7144feb0d10dce709e0e71'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:03.179762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2726112c-0027-4c4e-b9ef-9d157e561a10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607e48841d7144feb0d10dce709e0e71'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:03.191845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1014/2044] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:02 2022] DELETE /identity/v3/users/607e48841d7144feb0d10dce709e0e71 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:03.503785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:03.506108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:03.519622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07c314-b7ea-430a-9c34-6a6f38cd6679 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:03.577801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:03.578678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:03.578953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:03.579219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] PATH_INFO: `/v3/projects/64bbc33eaf0f4e0299df24427f42653b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:03.580605 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=64bbc33eaf0f4e0299df24427f42653b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:03.582481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:03.632685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64bbc33eaf0f4e0299df24427f42653b', 'request_id': 'req-8ca67603-a979-41d8-8475-6b4eb427a65c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:03.642618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64bbc33eaf0f4e0299df24427f42653b', 'request_id': 'req-8ca67603-a979-41d8-8475-6b4eb427a65c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:03.648782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64bbc33eaf0f4e0299df24427f42653b', 'request_id': 'req-8ca67603-a979-41d8-8475-6b4eb427a65c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:03.658745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] The token cache is being invalidate because project 64bbc33eaf0f4e0299df24427f42653b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:03.659022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:03.659770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8ca67603-a979-41d8-8475-6b4eb427a65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:03.661462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1031/2045] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:03 2022] DELETE /identity/v3/projects/64bbc33eaf0f4e0299df24427f42653b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:03.666550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea947e-08b4-46bb-9924-6e8fb42dd19d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:03.667067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea947e-08b4-46bb-9924-6e8fb42dd19d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:03.667159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea947e-08b4-46bb-9924-6e8fb42dd19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:03.690208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-18ea947e-08b4-46bb-9924-6e8fb42dd19d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:03.691407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-18ea947e-08b4-46bb-9924-6e8fb42dd19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:03.751000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1015/2046] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:03.759297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313a926-4b69-4d85-af13-e6f9ec834de7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:03.759577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313a926-4b69-4d85-af13-e6f9ec834de7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:03.759847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313a926-4b69-4d85-af13-e6f9ec834de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:03.778929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b313a926-4b69-4d85-af13-e6f9ec834de7 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:03.780190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b313a926-4b69-4d85-af13-e6f9ec834de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:03.788853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1032/2047] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:03.793190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea947e-08b4-46bb-9924-6e8fb42dd19d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:03.802940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea947e-08b4-46bb-9924-6e8fb42dd19d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:03.807934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa082a6-ae4f-4abf-90d2-9c356a5e1262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:03.808635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa082a6-ae4f-4abf-90d2-9c356a5e1262 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:03.808916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa082a6-ae4f-4abf-90d2-9c356a5e1262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:03.809187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa082a6-ae4f-4abf-90d2-9c356a5e1262 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:03.809934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa082a6-ae4f-4abf-90d2-9c356a5e1262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:45:03.810235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa082a6-ae4f-4abf-90d2-9c356a5e1262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:03.811856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa082a6-ae4f-4abf-90d2-9c356a5e1262 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:03.823973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1016/2048] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:45:03.828880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313a926-4b69-4d85-af13-e6f9ec834de7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:03.838092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313a926-4b69-4d85-af13-e6f9ec834de7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:03.842817 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69511e19-8bef-4173-9227-7069b09725a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:03.843468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69511e19-8bef-4173-9227-7069b09725a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:03.843746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69511e19-8bef-4173-9227-7069b09725a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:03.844014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69511e19-8bef-4173-9227-7069b09725a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:03.844728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69511e19-8bef-4173-9227-7069b09725a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:03.847116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69511e19-8bef-4173-9227-7069b09725a8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:03.874367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1033/2049] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:03 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:03.878803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa082a6-ae4f-4abf-90d2-9c356a5e1262 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:03.889401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa082a6-ae4f-4abf-90d2-9c356a5e1262 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:03.945694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473e88-9437-4822-8058-5210297cc778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:03.946369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473e88-9437-4822-8058-5210297cc778 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:03.946650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473e88-9437-4822-8058-5210297cc778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:03.946922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473e88-9437-4822-8058-5210297cc778 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:03.947601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b473e88-9437-4822-8058-5210297cc778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:03.949557 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b473e88-9437-4822-8058-5210297cc778 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:03.973821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1017/2050] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:03 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:45:03.975011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69511e19-8bef-4173-9227-7069b09725a8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:03.987933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69511e19-8bef-4173-9227-7069b09725a8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:03.992960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a5cf7-99c3-420d-b49e-36b5634fa72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:03.993617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a5cf7-99c3-420d-b49e-36b5634fa72e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:03.993898 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a5cf7-99c3-420d-b49e-36b5634fa72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:03.994173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a5cf7-99c3-420d-b49e-36b5634fa72e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:03.994906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a5cf7-99c3-420d-b49e-36b5634fa72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:03.996534 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a5cf7-99c3-420d-b49e-36b5634fa72e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:04.008204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1034/2051] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:45:04.012098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473e88-9437-4822-8058-5210297cc778 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:04.025310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473e88-9437-4822-8058-5210297cc778 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:04.032163 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87a17c-336a-49ba-9cdb-b9e403386b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:04.033015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87a17c-336a-49ba-9cdb-b9e403386b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.033299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87a17c-336a-49ba-9cdb-b9e403386b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.033572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87a17c-336a-49ba-9cdb-b9e403386b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.034428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87a17c-336a-49ba-9cdb-b9e403386b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:04.036468 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87a17c-336a-49ba-9cdb-b9e403386b57 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:04.047956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1018/2052] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:04.052596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a5cf7-99c3-420d-b49e-36b5634fa72e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:04.063531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a5cf7-99c3-420d-b49e-36b5634fa72e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:04.069266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870aafd0-10b6-4ce8-af7c-4282a44ee2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:04.070160 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aafd0-10b6-4ce8-af7c-4282a44ee2de admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.070554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aafd0-10b6-4ce8-af7c-4282a44ee2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.070711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aafd0-10b6-4ce8-af7c-4282a44ee2de admin admin] PATH_INFO: `/v3/projects/7f644f86133642d8af0b8524e4ef9276/users/d7d54aa8324d4e0aab39aa1b2f1fd362/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.086522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870aafd0-10b6-4ce8-af7c-4282a44ee2de admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f644f86133642d8af0b8524e4ef9276, user_id=d7d54aa8324d4e0aab39aa1b2f1fd362, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:04.089295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870aafd0-10b6-4ce8-af7c-4282a44ee2de admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:04.103410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1035/2053] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:04 2022] PUT /identity/v3/projects/7f644f86133642d8af0b8524e4ef9276/users/d7d54aa8324d4e0aab39aa1b2f1fd362/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:04.115661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9883916-f4f8-4252-9ddd-b6ecf4687067 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.115964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9883916-f4f8-4252-9ddd-b6ecf4687067 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.116263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9883916-f4f8-4252-9ddd-b6ecf4687067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.144550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f9883916-f4f8-4252-9ddd-b6ecf4687067 None None] MFA Rules not processed for user `d7d54aa8324d4e0aab39aa1b2f1fd362`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:04.145761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f9883916-f4f8-4252-9ddd-b6ecf4687067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:04.213614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1019/2054] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:04.227943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870aafd0-10b6-4ce8-af7c-4282a44ee2de admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:04.300344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870aafd0-10b6-4ce8-af7c-4282a44ee2de admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:04.305281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1d83e-b230-4f43-bde4-4d333e9a5412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:04.306014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1d83e-b230-4f43-bde4-4d333e9a5412 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.306291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1d83e-b230-4f43-bde4-4d333e9a5412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.306560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1d83e-b230-4f43-bde4-4d333e9a5412 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.307269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1d83e-b230-4f43-bde4-4d333e9a5412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:04.309246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1d83e-b230-4f43-bde4-4d333e9a5412 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:04.336246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1036/2055] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:04 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:04.340469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9883916-f4f8-4252-9ddd-b6ecf4687067 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:04.350855 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9883916-f4f8-4252-9ddd-b6ecf4687067 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:04.407613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c3510-52af-489b-9fa2-007cc2cce22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:04.408483 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c3510-52af-489b-9fa2-007cc2cce22b admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.408795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c3510-52af-489b-9fa2-007cc2cce22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.409065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c3510-52af-489b-9fa2-007cc2cce22b admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.409766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c3510-52af-489b-9fa2-007cc2cce22b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:04.412984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c3510-52af-489b-9fa2-007cc2cce22b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:04.434443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1020/2056] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:04 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:04.435744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1d83e-b230-4f43-bde4-4d333e9a5412 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:04.445921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1d83e-b230-4f43-bde4-4d333e9a5412 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:04.451402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10990719-f195-41b0-b8f0-b67141c1c8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:04.452090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10990719-f195-41b0-b8f0-b67141c1c8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.452373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10990719-f195-41b0-b8f0-b67141c1c8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.452683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10990719-f195-41b0-b8f0-b67141c1c8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.453452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10990719-f195-41b0-b8f0-b67141c1c8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:04.455226 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10990719-f195-41b0-b8f0-b67141c1c8f9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:04.466585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1037/2057] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:04.471004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c3510-52af-489b-9fa2-007cc2cce22b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:04.481627 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c3510-52af-489b-9fa2-007cc2cce22b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:04.487724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcff46-383f-4991-8824-a2f827924825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:04.487724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcff46-383f-4991-8824-a2f827924825 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.487724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcff46-383f-4991-8824-a2f827924825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.488171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcff46-383f-4991-8824-a2f827924825 admin admin] PATH_INFO: `/v3/projects/699fd94985b948fd8c0293b2b3f95cb6/users/560a06baa8af4559bfd3806a8ff88c86/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.505489 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcff46-383f-4991-8824-a2f827924825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=699fd94985b948fd8c0293b2b3f95cb6, user_id=560a06baa8af4559bfd3806a8ff88c86, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:04.511058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcff46-383f-4991-8824-a2f827924825 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:04.538852 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1021/2058] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:04 2022] PUT /identity/v3/projects/699fd94985b948fd8c0293b2b3f95cb6/users/560a06baa8af4559bfd3806a8ff88c86/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:04.542084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10990719-f195-41b0-b8f0-b67141c1c8f9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:04.565822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10990719-f195-41b0-b8f0-b67141c1c8f9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:04.651168 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6d279-d7a2-4f2f-a513-278f9ae82b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:04.651866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6d279-d7a2-4f2f-a513-278f9ae82b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.652148 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6d279-d7a2-4f2f-a513-278f9ae82b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.652413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6d279-d7a2-4f2f-a513-278f9ae82b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.653163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6d279-d7a2-4f2f-a513-278f9ae82b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:04.654745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6d279-d7a2-4f2f-a513-278f9ae82b23 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:04.664680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1038/2059] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:04.668896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcff46-383f-4991-8824-a2f827924825 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:04.679973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcff46-383f-4991-8824-a2f827924825 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:04.684724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28f073-3622-45da-ba74-238a731fd88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:04.685613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28f073-3622-45da-ba74-238a731fd88d admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.685886 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28f073-3622-45da-ba74-238a731fd88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.686152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28f073-3622-45da-ba74-238a731fd88d admin admin] PATH_INFO: `/v3/projects/699fd94985b948fd8c0293b2b3f95cb6/users/560a06baa8af4559bfd3806a8ff88c86/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.688139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28f073-3622-45da-ba74-238a731fd88d admin admin] RBAC: Authorizing `identity:create_grant(project_id=699fd94985b948fd8c0293b2b3f95cb6, user_id=560a06baa8af4559bfd3806a8ff88c86, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:04.691233 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28f073-3622-45da-ba74-238a731fd88d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:04.702006 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1022/2060] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:04 2022] PUT /identity/v3/projects/699fd94985b948fd8c0293b2b3f95cb6/users/560a06baa8af4559bfd3806a8ff88c86/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:04.712564 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e6c06-8fbc-4945-b992-b217c37bc03f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.712879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e6c06-8fbc-4945-b992-b217c37bc03f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.713167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e6c06-8fbc-4945-b992-b217c37bc03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.740563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b09e6c06-8fbc-4945-b992-b217c37bc03f None None] MFA Rules not processed for user `560a06baa8af4559bfd3806a8ff88c86`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:04.741783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b09e6c06-8fbc-4945-b992-b217c37bc03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:04.813755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1039/2061] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:04.824960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28f073-3622-45da-ba74-238a731fd88d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:04.827035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8e28f073-3622-45da-ba74-238a731fd88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:04.881619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28f073-3622-45da-ba74-238a731fd88d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:04.885538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2915f-3551-4fc5-9974-853af20d6424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:04.886183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2915f-3551-4fc5-9974-853af20d6424 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:04.886497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2915f-3551-4fc5-9974-853af20d6424 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:04.886738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2915f-3551-4fc5-9974-853af20d6424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:04.896167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2915f-3551-4fc5-9974-853af20d6424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:04.897633 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2915f-3551-4fc5-9974-853af20d6424 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:04.906926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1023/2062] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:05.535279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e6c06-8fbc-4945-b992-b217c37bc03f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:05.599588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e6c06-8fbc-4945-b992-b217c37bc03f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:05.604278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:05.605199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:05.605478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:05.605746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] PATH_INFO: `/v3/users/d7d54aa8324d4e0aab39aa1b2f1fd362` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:05.606964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d54aa8324d4e0aab39aa1b2f1fd362)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:05.609002 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:05.662902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d54aa8324d4e0aab39aa1b2f1fd362'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:05.663630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d54aa8324d4e0aab39aa1b2f1fd362'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:05.683151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d54aa8324d4e0aab39aa1b2f1fd362'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:05.686209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d54aa8324d4e0aab39aa1b2f1fd362'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:05.694385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d54aa8324d4e0aab39aa1b2f1fd362'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:05.703935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1040/2063] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:05 2022] DELETE /identity/v3/users/d7d54aa8324d4e0aab39aa1b2f1fd362 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:05.706878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e7022-2058-47a5-8621-bcf20852f7b3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:05.707364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e7022-2058-47a5-8621-bcf20852f7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:05.707442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e7022-2058-47a5-8621-bcf20852f7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:05.726348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-800e7022-2058-47a5-8621-bcf20852f7b3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:05.727626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-800e7022-2058-47a5-8621-bcf20852f7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:05.783037 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1024/2064] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:05.796928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:05.799539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:05.859925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1798740-d885-4e3c-9ae4-8f6f1756037b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:05.864112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a68002-5a0d-4593-bed6-84ccda3f177d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:05.864786 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a68002-5a0d-4593-bed6-84ccda3f177d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:05.865065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a68002-5a0d-4593-bed6-84ccda3f177d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:05.865334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a68002-5a0d-4593-bed6-84ccda3f177d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:05.874839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a68002-5a0d-4593-bed6-84ccda3f177d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:05.876255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a68002-5a0d-4593-bed6-84ccda3f177d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:05.885779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1041/2065] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:06.147788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e7022-2058-47a5-8621-bcf20852f7b3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:06.149553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-800e7022-2058-47a5-8621-bcf20852f7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:06.163388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e7022-2058-47a5-8621-bcf20852f7b3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:06.168459 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:06.170258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:06.170754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:06.171211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] PATH_INFO: `/v3/projects/7f644f86133642d8af0b8524e4ef9276` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:06.173731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f644f86133642d8af0b8524e4ef9276)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:06.176376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:06.236662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f644f86133642d8af0b8524e4ef9276', 'request_id': 'req-edafe749-d57f-412b-a0f7-1338d17c25f8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:06.245471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f644f86133642d8af0b8524e4ef9276', 'request_id': 'req-edafe749-d57f-412b-a0f7-1338d17c25f8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:06.252526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f644f86133642d8af0b8524e4ef9276', 'request_id': 'req-edafe749-d57f-412b-a0f7-1338d17c25f8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:06.264617 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] The token cache is being invalidate because project 7f644f86133642d8af0b8524e4ef9276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:06.264897 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:06.265558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:06.267236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1025/2066] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:06 2022] DELETE /identity/v3/projects/7f644f86133642d8af0b8524e4ef9276 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:06.271530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a68002-5a0d-4593-bed6-84ccda3f177d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:06.281113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a68002-5a0d-4593-bed6-84ccda3f177d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:06.346239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:06.347122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:06.347400 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:06.347671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] PATH_INFO: `/v3/users/560a06baa8af4559bfd3806a8ff88c86` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:06.348933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=560a06baa8af4559bfd3806a8ff88c86)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:06.350858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:06.399620 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '560a06baa8af4559bfd3806a8ff88c86'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:06.400298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '560a06baa8af4559bfd3806a8ff88c86'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:06.411971 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '560a06baa8af4559bfd3806a8ff88c86'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:06.419708 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '560a06baa8af4559bfd3806a8ff88c86'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:06.426351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e824d6ec-d4a7-49d3-a372-ddb3abd6ed08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '560a06baa8af4559bfd3806a8ff88c86'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:06.432081 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1042/2067] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:06 2022] DELETE /identity/v3/users/560a06baa8af4559bfd3806a8ff88c86 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:06.932020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:06.933778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:06.956320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafe749-d57f-412b-a0f7-1338d17c25f8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:07.016109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:07.017531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.017816 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.018094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] PATH_INFO: `/v3/projects/699fd94985b948fd8c0293b2b3f95cb6` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.019330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=699fd94985b948fd8c0293b2b3f95cb6)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:07.021306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:07.066369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699fd94985b948fd8c0293b2b3f95cb6', 'request_id': 'req-97aebaad-21a1-4616-8b63-827129cc0b9a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:07.075188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699fd94985b948fd8c0293b2b3f95cb6', 'request_id': 'req-97aebaad-21a1-4616-8b63-827129cc0b9a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:07.082392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699fd94985b948fd8c0293b2b3f95cb6', 'request_id': 'req-97aebaad-21a1-4616-8b63-827129cc0b9a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:07.092321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] The token cache is being invalidate because project 699fd94985b948fd8c0293b2b3f95cb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:07.092631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:07.093292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-97aebaad-21a1-4616-8b63-827129cc0b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:07.094851 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1026/2068] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:06 2022] DELETE /identity/v3/projects/699fd94985b948fd8c0293b2b3f95cb6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:07.099890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161dc6-448d-4001-967d-4ad32cfe832c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.100359 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161dc6-448d-4001-967d-4ad32cfe832c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.100465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161dc6-448d-4001-967d-4ad32cfe832c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.119813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d9161dc6-448d-4001-967d-4ad32cfe832c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:07.121074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d9161dc6-448d-4001-967d-4ad32cfe832c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:07.178079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1043/2069] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:07.185511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468afe-a999-4f7b-9122-0651488b29a2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.185865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468afe-a999-4f7b-9122-0651488b29a2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.186199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468afe-a999-4f7b-9122-0651488b29a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.206090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-96468afe-a999-4f7b-9122-0651488b29a2 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:07.207370 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-96468afe-a999-4f7b-9122-0651488b29a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:07.215663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1027/2070] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:07.220283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9161dc6-448d-4001-967d-4ad32cfe832c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:07.231616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9161dc6-448d-4001-967d-4ad32cfe832c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:07.236350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ba643-2676-4525-b060-2ef645a4ffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:07.237048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ba643-2676-4525-b060-2ef645a4ffb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.237323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ba643-2676-4525-b060-2ef645a4ffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.237591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ba643-2676-4525-b060-2ef645a4ffb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.239395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ba643-2676-4525-b060-2ef645a4ffb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:45:07.239395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ba643-2676-4525-b060-2ef645a4ffb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:07.240202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ba643-2676-4525-b060-2ef645a4ffb3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:07.252308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1044/2071] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:45:07.256814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96468afe-a999-4f7b-9122-0651488b29a2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:07.266462 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96468afe-a999-4f7b-9122-0651488b29a2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:07.272346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c11a5-4fda-482c-baa1-2728fe76afd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:07.273048 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c11a5-4fda-482c-baa1-2728fe76afd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.273321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c11a5-4fda-482c-baa1-2728fe76afd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.273612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c11a5-4fda-482c-baa1-2728fe76afd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.274291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c11a5-4fda-482c-baa1-2728fe76afd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:07.276343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c11a5-4fda-482c-baa1-2728fe76afd2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:07.309740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1028/2072] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:07 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:07.314391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ba643-2676-4525-b060-2ef645a4ffb3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:07.327760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ba643-2676-4525-b060-2ef645a4ffb3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:07.394147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47836c08-2096-4c7e-bea8-98933c7b5831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:07.394936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47836c08-2096-4c7e-bea8-98933c7b5831 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.395208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47836c08-2096-4c7e-bea8-98933c7b5831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.395473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47836c08-2096-4c7e-bea8-98933c7b5831 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.396380 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47836c08-2096-4c7e-bea8-98933c7b5831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:07.398429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47836c08-2096-4c7e-bea8-98933c7b5831 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:07.424282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1045/2073] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:07 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:07.425021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c11a5-4fda-482c-baa1-2728fe76afd2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:07.436705 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c11a5-4fda-482c-baa1-2728fe76afd2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:07.443810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34741e2-eba1-4367-a45a-be422dff64c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:07.445393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34741e2-eba1-4367-a45a-be422dff64c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.445393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34741e2-eba1-4367-a45a-be422dff64c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.445393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34741e2-eba1-4367-a45a-be422dff64c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.446455 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34741e2-eba1-4367-a45a-be422dff64c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:07.448455 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34741e2-eba1-4367-a45a-be422dff64c2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:07.461373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1029/2074] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:07.465665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47836c08-2096-4c7e-bea8-98933c7b5831 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:07.477077 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47836c08-2096-4c7e-bea8-98933c7b5831 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:07.481719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e40dc-1295-47ae-b816-1befc4fb308c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:07.482388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e40dc-1295-47ae-b816-1befc4fb308c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.482678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e40dc-1295-47ae-b816-1befc4fb308c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.482947 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e40dc-1295-47ae-b816-1befc4fb308c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.483705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e40dc-1295-47ae-b816-1befc4fb308c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:07.485305 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e40dc-1295-47ae-b816-1befc4fb308c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:07.495696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1046/2075] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:07.500075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34741e2-eba1-4367-a45a-be422dff64c2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:07.510897 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34741e2-eba1-4367-a45a-be422dff64c2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:07.515664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbc217-aae6-49d8-ac4f-d9580be80a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:07.516593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbc217-aae6-49d8-ac4f-d9580be80a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.516879 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbc217-aae6-49d8-ac4f-d9580be80a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.517191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbc217-aae6-49d8-ac4f-d9580be80a5e admin admin] PATH_INFO: `/v3/projects/e9ea298310964ff9ba31f06396998bc2/users/967e91053d4344fd90e37dea693fe75e/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.532226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbc217-aae6-49d8-ac4f-d9580be80a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ea298310964ff9ba31f06396998bc2, user_id=967e91053d4344fd90e37dea693fe75e, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:07.535184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbc217-aae6-49d8-ac4f-d9580be80a5e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:07.549443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1030/2076] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:07 2022] PUT /identity/v3/projects/e9ea298310964ff9ba31f06396998bc2/users/967e91053d4344fd90e37dea693fe75e/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:07.560916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4130ec7-77a5-47e0-82b7-18cfb84d6616 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.561220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4130ec7-77a5-47e0-82b7-18cfb84d6616 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.561490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4130ec7-77a5-47e0-82b7-18cfb84d6616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.589162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c4130ec7-77a5-47e0-82b7-18cfb84d6616 None None] MFA Rules not processed for user `967e91053d4344fd90e37dea693fe75e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:07.590531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c4130ec7-77a5-47e0-82b7-18cfb84d6616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:07.665270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1047/2077] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:45:07.677070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbc217-aae6-49d8-ac4f-d9580be80a5e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:07.679084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5cdbc217-aae6-49d8-ac4f-d9580be80a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:07.747211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbc217-aae6-49d8-ac4f-d9580be80a5e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:07.751753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f5f77-6063-44af-9761-840a639c4d7e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6552c41f1c654ef5bfdd940e3b807dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:07.752802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f5f77-6063-44af-9761-840a639c4d7e service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:07.753207 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f5f77-6063-44af-9761-840a639c4d7e service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:07.753576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f5f77-6063-44af-9761-840a639c4d7e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:07.764024 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f5f77-6063-44af-9761-840a639c4d7e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:07.765934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f5f77-6063-44af-9761-840a639c4d7e service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:07.776623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1031/2078] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:13.098042 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4130ec7-77a5-47e0-82b7-18cfb84d6616 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:13.100436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c4130ec7-77a5-47e0-82b7-18cfb84d6616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:13.201832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4130ec7-77a5-47e0-82b7-18cfb84d6616 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:13.215775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:13.216945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:13.217446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:13.217900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] PATH_INFO: `/v3/users/e37682766bf840309e530b3f084074ee` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:13.219413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e37682766bf840309e530b3f084074ee)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:13.222109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:13.297705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e37682766bf840309e530b3f084074ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:13.298252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e37682766bf840309e530b3f084074ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:13.316711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e37682766bf840309e530b3f084074ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:13.327245 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e37682766bf840309e530b3f084074ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:13.341176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e37682766bf840309e530b3f084074ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:13.348588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1048/2079] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:13 2022] DELETE /identity/v3/users/e37682766bf840309e530b3f084074ee => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:13.354202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c261906-794f-47ec-b499-b86a710c1847 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:13.354812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c261906-794f-47ec-b499-b86a710c1847 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:13.354812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c261906-794f-47ec-b499-b86a710c1847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:13.378522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0c261906-794f-47ec-b499-b86a710c1847 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:13.379934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0c261906-794f-47ec-b499-b86a710c1847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:13.446486 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1032/2080] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:13.458309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:13.459944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:13.533746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873faded-3715-4507-a636-d93b9e1c53fa admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:13.538546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0fb9a-9e2b-4229-ae88-9d35089311c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:13.539318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0fb9a-9e2b-4229-ae88-9d35089311c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:13.539654 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0fb9a-9e2b-4229-ae88-9d35089311c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:13.539930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0fb9a-9e2b-4229-ae88-9d35089311c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:13.551038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0fb9a-9e2b-4229-ae88-9d35089311c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:13.552442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0fb9a-9e2b-4229-ae88-9d35089311c5 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:13.567668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1049/2081] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:13.873254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c261906-794f-47ec-b499-b86a710c1847 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:13.874782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0c261906-794f-47ec-b499-b86a710c1847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:13.886944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c261906-794f-47ec-b499-b86a710c1847 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:13.891600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:13.892775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:13.893125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:13.893465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] PATH_INFO: `/v3/projects/e96c11f4daeb48e7af7aee6e30e58661` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:13.894750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96c11f4daeb48e7af7aee6e30e58661)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:13.896791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:13.955295 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96c11f4daeb48e7af7aee6e30e58661', 'request_id': 'req-2b35709e-6f5d-4768-9e8b-f594a94cc789'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:13.964212 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96c11f4daeb48e7af7aee6e30e58661', 'request_id': 'req-2b35709e-6f5d-4768-9e8b-f594a94cc789'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:13.971269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96c11f4daeb48e7af7aee6e30e58661', 'request_id': 'req-2b35709e-6f5d-4768-9e8b-f594a94cc789'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:13.984247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] The token cache is being invalidate because project e96c11f4daeb48e7af7aee6e30e58661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:13.984580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:13.985238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:13.987058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1033/2082] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:13 2022] DELETE /identity/v3/projects/e96c11f4daeb48e7af7aee6e30e58661 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:13.991057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0fb9a-9e2b-4229-ae88-9d35089311c5 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:14.000871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0fb9a-9e2b-4229-ae88-9d35089311c5 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:14.058390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:14.059267 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:14.059541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:14.059782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] PATH_INFO: `/v3/users/8f27a0449586440eb9c3f238a8c6195d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:14.061028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f27a0449586440eb9c3f238a8c6195d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:14.062915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:14.110813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f27a0449586440eb9c3f238a8c6195d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:14.112423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f27a0449586440eb9c3f238a8c6195d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:14.125061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f27a0449586440eb9c3f238a8c6195d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:14.132908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f27a0449586440eb9c3f238a8c6195d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:14.139772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-58f39cfc-2024-4ea5-a368-1aca7a087425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f27a0449586440eb9c3f238a8c6195d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:14.145587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1050/2083] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:13 2022] DELETE /identity/v3/users/8f27a0449586440eb9c3f238a8c6195d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:14.445371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:14.446941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:14.463469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35709e-6f5d-4768-9e8b-f594a94cc789 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:14.520904 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:14.521768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:14.522044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:14.522311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] PATH_INFO: `/v3/projects/696427417d7049359bf65537c50374e4` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:14.523493 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] RBAC: Authorizing `identity:delete_project(project_id=696427417d7049359bf65537c50374e4)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:14.525420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:14.571667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696427417d7049359bf65537c50374e4', 'request_id': 'req-25a7693e-b2bc-4c54-b693-ceae9273f78b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:14.581527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696427417d7049359bf65537c50374e4', 'request_id': 'req-25a7693e-b2bc-4c54-b693-ceae9273f78b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:14.587850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696427417d7049359bf65537c50374e4', 'request_id': 'req-25a7693e-b2bc-4c54-b693-ceae9273f78b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:14.598316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] The token cache is being invalidate because project 696427417d7049359bf65537c50374e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:14.598842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:14.599473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25a7693e-b2bc-4c54-b693-ceae9273f78b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:14.601080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1034/2084] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:14 2022] DELETE /identity/v3/projects/696427417d7049359bf65537c50374e4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:14.606362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7ebd8-5c03-4711-9b53-58b80dfe916e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:14.606643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7ebd8-5c03-4711-9b53-58b80dfe916e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:14.606966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7ebd8-5c03-4711-9b53-58b80dfe916e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:14.624888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1cd7ebd8-5c03-4711-9b53-58b80dfe916e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:14.626078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1cd7ebd8-5c03-4711-9b53-58b80dfe916e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:14.695024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1051/2085] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:14.708996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e129a3-8748-40b6-846f-3c799d9bc07a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:14.709343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e129a3-8748-40b6-846f-3c799d9bc07a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:14.709621 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e129a3-8748-40b6-846f-3c799d9bc07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:14.728317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-05e129a3-8748-40b6-846f-3c799d9bc07a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:14.729554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-05e129a3-8748-40b6-846f-3c799d9bc07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:14.738974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1035/2086] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:14.743154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7ebd8-5c03-4711-9b53-58b80dfe916e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:14.753111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7ebd8-5c03-4711-9b53-58b80dfe916e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:14.758132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754076bb-ee67-456b-b598-4be8f09eb60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:14.758799 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754076bb-ee67-456b-b598-4be8f09eb60e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:14.759076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754076bb-ee67-456b-b598-4be8f09eb60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:14.759345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754076bb-ee67-456b-b598-4be8f09eb60e admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:14.760133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754076bb-ee67-456b-b598-4be8f09eb60e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:45:14.760465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754076bb-ee67-456b-b598-4be8f09eb60e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:14.762161 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754076bb-ee67-456b-b598-4be8f09eb60e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:14.774261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1052/2087] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:45:14.778921 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e129a3-8748-40b6-846f-3c799d9bc07a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:14.792166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e129a3-8748-40b6-846f-3c799d9bc07a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:14.800821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d1c0b-ada8-45de-a5b1-50f4216e650b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:14.801823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d1c0b-ada8-45de-a5b1-50f4216e650b admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:14.802217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d1c0b-ada8-45de-a5b1-50f4216e650b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:14.802707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d1c0b-ada8-45de-a5b1-50f4216e650b admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:14.803875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d1c0b-ada8-45de-a5b1-50f4216e650b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:14.806846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d1c0b-ada8-45de-a5b1-50f4216e650b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:14.839880 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1036/2088] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:14 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:14.844165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754076bb-ee67-456b-b598-4be8f09eb60e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:14.854740 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754076bb-ee67-456b-b598-4be8f09eb60e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:14.910501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057a76a-7fd5-42f6-9ff0-adb845e9426d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:14.911267 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057a76a-7fd5-42f6-9ff0-adb845e9426d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:14.911603 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057a76a-7fd5-42f6-9ff0-adb845e9426d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:14.916404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057a76a-7fd5-42f6-9ff0-adb845e9426d admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:14.916404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057a76a-7fd5-42f6-9ff0-adb845e9426d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:14.916404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057a76a-7fd5-42f6-9ff0-adb845e9426d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:14.937999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1053/2089] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:14 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:14.943492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d1c0b-ada8-45de-a5b1-50f4216e650b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:14.954174 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d1c0b-ada8-45de-a5b1-50f4216e650b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:14.959674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedeed3-ec60-40a6-81b4-934eb7c2e273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:14.960363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedeed3-ec60-40a6-81b4-934eb7c2e273 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:14.960697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedeed3-ec60-40a6-81b4-934eb7c2e273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:14.960952 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedeed3-ec60-40a6-81b4-934eb7c2e273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:14.961678 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedeed3-ec60-40a6-81b4-934eb7c2e273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:14.963282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedeed3-ec60-40a6-81b4-934eb7c2e273 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:14.976291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1037/2090] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:14.980877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057a76a-7fd5-42f6-9ff0-adb845e9426d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:14.994715 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057a76a-7fd5-42f6-9ff0-adb845e9426d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:15.000674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d34b10-e4c5-49ad-ade4-a5e84a8657fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:15.001751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d34b10-e4c5-49ad-ade4-a5e84a8657fb admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.002082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d34b10-e4c5-49ad-ade4-a5e84a8657fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.002388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d34b10-e4c5-49ad-ade4-a5e84a8657fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.003096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d34b10-e4c5-49ad-ade4-a5e84a8657fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:15.004734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d34b10-e4c5-49ad-ade4-a5e84a8657fb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:15.016636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1054/2091] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:15.019572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedeed3-ec60-40a6-81b4-934eb7c2e273 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:15.034571 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedeed3-ec60-40a6-81b4-934eb7c2e273 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:15.039803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b217ea-8dc7-493c-97d6-49366f893cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:15.040743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b217ea-8dc7-493c-97d6-49366f893cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.041058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b217ea-8dc7-493c-97d6-49366f893cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.041284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b217ea-8dc7-493c-97d6-49366f893cfc admin admin] PATH_INFO: `/v3/projects/3d48106b3c804a4fa544dc691a680195/users/65b8c9a111ea4fb299f28281bc309f72/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.057058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b217ea-8dc7-493c-97d6-49366f893cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d48106b3c804a4fa544dc691a680195, user_id=65b8c9a111ea4fb299f28281bc309f72, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:15.059866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b217ea-8dc7-493c-97d6-49366f893cfc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:15.074008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1038/2092] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:15 2022] PUT /identity/v3/projects/3d48106b3c804a4fa544dc691a680195/users/65b8c9a111ea4fb299f28281bc309f72/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:15.086112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6050f73-07dc-4064-ae87-74f84562038a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.087962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6050f73-07dc-4064-ae87-74f84562038a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.087962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6050f73-07dc-4064-ae87-74f84562038a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.115363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d6050f73-07dc-4064-ae87-74f84562038a None None] MFA Rules not processed for user `65b8c9a111ea4fb299f28281bc309f72`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:15.116619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6050f73-07dc-4064-ae87-74f84562038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:15.180884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1055/2093] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:15.185296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b217ea-8dc7-493c-97d6-49366f893cfc admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:15.252143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b217ea-8dc7-493c-97d6-49366f893cfc admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:15.261239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e7fec-d665-4fba-bcd5-cd777f6174a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:15.262128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e7fec-d665-4fba-bcd5-cd777f6174a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.262506 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e7fec-d665-4fba-bcd5-cd777f6174a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.262874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e7fec-d665-4fba-bcd5-cd777f6174a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.263892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e7fec-d665-4fba-bcd5-cd777f6174a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:15.266884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e7fec-d665-4fba-bcd5-cd777f6174a0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:15.295265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1039/2094] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:15 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:15.299944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6050f73-07dc-4064-ae87-74f84562038a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:15.314994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6050f73-07dc-4064-ae87-74f84562038a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:15.373610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37d886-e405-406a-bbf8-49dcf46bd538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:15.374212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37d886-e405-406a-bbf8-49dcf46bd538 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.374508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37d886-e405-406a-bbf8-49dcf46bd538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.374763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37d886-e405-406a-bbf8-49dcf46bd538 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.375450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37d886-e405-406a-bbf8-49dcf46bd538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:15.377436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37d886-e405-406a-bbf8-49dcf46bd538 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:15.398164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1056/2095] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:15 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:15.399968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e7fec-d665-4fba-bcd5-cd777f6174a0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:15.411894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e7fec-d665-4fba-bcd5-cd777f6174a0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:15.416986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917e878-1263-40e4-a7a8-b838350a0aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:15.417696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917e878-1263-40e4-a7a8-b838350a0aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.417974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917e878-1263-40e4-a7a8-b838350a0aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.418245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917e878-1263-40e4-a7a8-b838350a0aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.419084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917e878-1263-40e4-a7a8-b838350a0aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:15.420700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917e878-1263-40e4-a7a8-b838350a0aa3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:15.432455 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1040/2096] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:15.436934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37d886-e405-406a-bbf8-49dcf46bd538 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:15.447768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37d886-e405-406a-bbf8-49dcf46bd538 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:15.453567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ef0b6-f869-44e7-88d7-e14172f6216d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:15.454778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ef0b6-f869-44e7-88d7-e14172f6216d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.455085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ef0b6-f869-44e7-88d7-e14172f6216d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.455361 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ef0b6-f869-44e7-88d7-e14172f6216d admin admin] PATH_INFO: `/v3/projects/23c6dd1b3ea34a07a5a81b18cf6dcd81/users/ac88cd03bbd642ce91b8ff3319e23f30/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.472194 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ef0b6-f869-44e7-88d7-e14172f6216d admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c6dd1b3ea34a07a5a81b18cf6dcd81, user_id=ac88cd03bbd642ce91b8ff3319e23f30, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:15.475887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ef0b6-f869-44e7-88d7-e14172f6216d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:15.495474 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1057/2097] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:15 2022] PUT /identity/v3/projects/23c6dd1b3ea34a07a5a81b18cf6dcd81/users/ac88cd03bbd642ce91b8ff3319e23f30/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:15.500212 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917e878-1263-40e4-a7a8-b838350a0aa3 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:15.510940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917e878-1263-40e4-a7a8-b838350a0aa3 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:15.568542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3c713-50ce-47bf-93b8-a304d10cc10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:15.569248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c713-50ce-47bf-93b8-a304d10cc10e admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.569542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c713-50ce-47bf-93b8-a304d10cc10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.569786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c713-50ce-47bf-93b8-a304d10cc10e admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.570536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3c713-50ce-47bf-93b8-a304d10cc10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:15.572172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3c713-50ce-47bf-93b8-a304d10cc10e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:15.583105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1041/2098] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:15.587471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ef0b6-f869-44e7-88d7-e14172f6216d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:15.598961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ef0b6-f869-44e7-88d7-e14172f6216d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:15.607290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbe0ea-4748-4b4d-95ff-1e49eae63547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:15.608103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbe0ea-4748-4b4d-95ff-1e49eae63547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.608367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbe0ea-4748-4b4d-95ff-1e49eae63547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.608663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbe0ea-4748-4b4d-95ff-1e49eae63547 admin admin] PATH_INFO: `/v3/projects/23c6dd1b3ea34a07a5a81b18cf6dcd81/users/ac88cd03bbd642ce91b8ff3319e23f30/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.610692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbe0ea-4748-4b4d-95ff-1e49eae63547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c6dd1b3ea34a07a5a81b18cf6dcd81, user_id=ac88cd03bbd642ce91b8ff3319e23f30, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:15.613809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbe0ea-4748-4b4d-95ff-1e49eae63547 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:15.641508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1058/2099] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:15 2022] PUT /identity/v3/projects/23c6dd1b3ea34a07a5a81b18cf6dcd81/users/ac88cd03bbd642ce91b8ff3319e23f30/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:15.645247 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241b03d-3172-4cb3-9a39-26dd7d373d34 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.645539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241b03d-3172-4cb3-9a39-26dd7d373d34 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.645819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241b03d-3172-4cb3-9a39-26dd7d373d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.675026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c241b03d-3172-4cb3-9a39-26dd7d373d34 None None] MFA Rules not processed for user `ac88cd03bbd642ce91b8ff3319e23f30`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:15.676220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c241b03d-3172-4cb3-9a39-26dd7d373d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:15.756352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1042/2100] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:15.767897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbe0ea-4748-4b4d-95ff-1e49eae63547 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:15.773223 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-14dbe0ea-4748-4b4d-95ff-1e49eae63547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:15.830706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbe0ea-4748-4b4d-95ff-1e49eae63547 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:15.834285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0199ec5-3310-4ed7-9958-70487b7f0a0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:15.834934 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0199ec5-3310-4ed7-9958-70487b7f0a0c service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:15.835212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0199ec5-3310-4ed7-9958-70487b7f0a0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:15.835485 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0199ec5-3310-4ed7-9958-70487b7f0a0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:15.844156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0199ec5-3310-4ed7-9958-70487b7f0a0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:15.845778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0199ec5-3310-4ed7-9958-70487b7f0a0c service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:15.862590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1059/2101] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:15.926188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241b03d-3172-4cb3-9a39-26dd7d373d34 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:15.997501 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241b03d-3172-4cb3-9a39-26dd7d373d34 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:16.002528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:16.003401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:16.003658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:16.003949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] PATH_INFO: `/v3/users/65b8c9a111ea4fb299f28281bc309f72` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:16.005362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b8c9a111ea4fb299f28281bc309f72)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:16.007263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:16.059698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b8c9a111ea4fb299f28281bc309f72'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.060442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b8c9a111ea4fb299f28281bc309f72'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.073517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b8c9a111ea4fb299f28281bc309f72'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.082420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b8c9a111ea4fb299f28281bc309f72'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.091328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b8c9a111ea4fb299f28281bc309f72'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.097538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1043/2102] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:15 2022] DELETE /identity/v3/users/65b8c9a111ea4fb299f28281bc309f72 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:16.102298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd3664-dd09-4165-bc42-567db67b1b72 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:16.102614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd3664-dd09-4165-bc42-567db67b1b72 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:16.102852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd3664-dd09-4165-bc42-567db67b1b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:16.120691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-09dd3664-dd09-4165-bc42-567db67b1b72 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:16.121878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-09dd3664-dd09-4165-bc42-567db67b1b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:16.183939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1060/2103] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:16.194318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:16.195813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:16.254612 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1354e-2aa7-4b43-a852-0e7f574627aa admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:16.258313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6c951-ec09-4216-8101-e1d6370e3677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:16.258976 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6c951-ec09-4216-8101-e1d6370e3677 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:16.259249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6c951-ec09-4216-8101-e1d6370e3677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:16.259529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6c951-ec09-4216-8101-e1d6370e3677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:16.269510 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6c951-ec09-4216-8101-e1d6370e3677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:16.270956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6c951-ec09-4216-8101-e1d6370e3677 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:16.280608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1044/2104] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:16.579835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd3664-dd09-4165-bc42-567db67b1b72 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:16.581559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-09dd3664-dd09-4165-bc42-567db67b1b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:16.595904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd3664-dd09-4165-bc42-567db67b1b72 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:16.606148 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:16.608979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:16.609393 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:16.609959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] PATH_INFO: `/v3/projects/3d48106b3c804a4fa544dc691a680195` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:16.611952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d48106b3c804a4fa544dc691a680195)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:16.613972 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:16.664692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d48106b3c804a4fa544dc691a680195', 'request_id': 'req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.673142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d48106b3c804a4fa544dc691a680195', 'request_id': 'req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.679611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d48106b3c804a4fa544dc691a680195', 'request_id': 'req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.689447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] The token cache is being invalidate because project 3d48106b3c804a4fa544dc691a680195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:16.689646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.690252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.691791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1061/2105] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:16 2022] DELETE /identity/v3/projects/3d48106b3c804a4fa544dc691a680195 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:16.695885 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6c951-ec09-4216-8101-e1d6370e3677 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:16.705915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6c951-ec09-4216-8101-e1d6370e3677 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:16.764028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:16.764957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:16.765240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:16.765515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] PATH_INFO: `/v3/users/ac88cd03bbd642ce91b8ff3319e23f30` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:16.766726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac88cd03bbd642ce91b8ff3319e23f30)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:16.768966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:16.819182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac88cd03bbd642ce91b8ff3319e23f30'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.819932 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac88cd03bbd642ce91b8ff3319e23f30'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.831763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac88cd03bbd642ce91b8ff3319e23f30'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.838124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac88cd03bbd642ce91b8ff3319e23f30'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.846433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7d9baacd-7165-4abc-8b01-7ef7e9f583ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac88cd03bbd642ce91b8ff3319e23f30'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:16.852393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1045/2106] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:16 2022] DELETE /identity/v3/users/ac88cd03bbd642ce91b8ff3319e23f30 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:17.150393 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:17.152449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:17.168475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af70f2c-7d7e-4db7-b8bb-d6b316321c00 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:17.228592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:17.229466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.229749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.230021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] PATH_INFO: `/v3/projects/23c6dd1b3ea34a07a5a81b18cf6dcd81` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.231224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c6dd1b3ea34a07a5a81b18cf6dcd81)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:17.233163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:17.279485 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c6dd1b3ea34a07a5a81b18cf6dcd81', 'request_id': 'req-0ce08420-5628-4637-9c39-4d01c0100c89'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:17.289984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c6dd1b3ea34a07a5a81b18cf6dcd81', 'request_id': 'req-0ce08420-5628-4637-9c39-4d01c0100c89'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:17.296293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c6dd1b3ea34a07a5a81b18cf6dcd81', 'request_id': 'req-0ce08420-5628-4637-9c39-4d01c0100c89'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:17.307053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] The token cache is being invalidate because project 23c6dd1b3ea34a07a5a81b18cf6dcd81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:17.307332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:17.307968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0ce08420-5628-4637-9c39-4d01c0100c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:17.309634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1062/2107] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:17 2022] DELETE /identity/v3/projects/23c6dd1b3ea34a07a5a81b18cf6dcd81 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:17.314801 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602354b1-b04c-4332-a011-e76bea48b9d0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.315144 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602354b1-b04c-4332-a011-e76bea48b9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.315424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602354b1-b04c-4332-a011-e76bea48b9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.335757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-602354b1-b04c-4332-a011-e76bea48b9d0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:17.337045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-602354b1-b04c-4332-a011-e76bea48b9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:17.397975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1046/2108] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:17.406032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961f3ab-1771-41da-99e2-27fe06f3f4c2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.406374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961f3ab-1771-41da-99e2-27fe06f3f4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.406653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961f3ab-1771-41da-99e2-27fe06f3f4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.426820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e961f3ab-1771-41da-99e2-27fe06f3f4c2 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:17.428043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e961f3ab-1771-41da-99e2-27fe06f3f4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:17.437302 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1063/2109] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:17.441443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602354b1-b04c-4332-a011-e76bea48b9d0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:17.452350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602354b1-b04c-4332-a011-e76bea48b9d0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:17.458179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3b7e9-e402-47e2-944c-fdb8c3b66ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:17.458847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b7e9-e402-47e2-944c-fdb8c3b66ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.459128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b7e9-e402-47e2-944c-fdb8c3b66ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.459398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b7e9-e402-47e2-944c-fdb8c3b66ea1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.460128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3b7e9-e402-47e2-944c-fdb8c3b66ea1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:45:17.460429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3b7e9-e402-47e2-944c-fdb8c3b66ea1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:17.462068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3b7e9-e402-47e2-944c-fdb8c3b66ea1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:17.474273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1047/2110] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:45:17.478554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961f3ab-1771-41da-99e2-27fe06f3f4c2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:17.488761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961f3ab-1771-41da-99e2-27fe06f3f4c2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:17.494164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5475a-b084-4cea-a5a0-b63fb658b63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:17.494827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5475a-b084-4cea-a5a0-b63fb658b63b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.495107 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5475a-b084-4cea-a5a0-b63fb658b63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.495374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5475a-b084-4cea-a5a0-b63fb658b63b admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.496059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5475a-b084-4cea-a5a0-b63fb658b63b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:17.497974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5475a-b084-4cea-a5a0-b63fb658b63b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:17.525324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1064/2111] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:17 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:17.529661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3b7e9-e402-47e2-944c-fdb8c3b66ea1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:17.539665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3b7e9-e402-47e2-944c-fdb8c3b66ea1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:17.599303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6aab69-62dc-4d90-b214-bc3dc6e2c4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:17.599984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6aab69-62dc-4d90-b214-bc3dc6e2c4db admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.600261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6aab69-62dc-4d90-b214-bc3dc6e2c4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.600537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6aab69-62dc-4d90-b214-bc3dc6e2c4db admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.601217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6aab69-62dc-4d90-b214-bc3dc6e2c4db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:17.603690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6aab69-62dc-4d90-b214-bc3dc6e2c4db admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:17.624884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1048/2112] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:17 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:45:17.628832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5475a-b084-4cea-a5a0-b63fb658b63b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:17.640144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5475a-b084-4cea-a5a0-b63fb658b63b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:17.648169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff63fc8-3b25-45d4-8dd7-20c48fd9aca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:17.649091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff63fc8-3b25-45d4-8dd7-20c48fd9aca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.649467 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff63fc8-3b25-45d4-8dd7-20c48fd9aca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.649823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff63fc8-3b25-45d4-8dd7-20c48fd9aca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.650921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff63fc8-3b25-45d4-8dd7-20c48fd9aca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:17.653381 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff63fc8-3b25-45d4-8dd7-20c48fd9aca1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:17.667779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1065/2113] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:17.672773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6aab69-62dc-4d90-b214-bc3dc6e2c4db admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:17.683647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6aab69-62dc-4d90-b214-bc3dc6e2c4db admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:17.689979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fc9cf-2787-426a-958c-a6e300b2acf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:17.690683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fc9cf-2787-426a-958c-a6e300b2acf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.690961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fc9cf-2787-426a-958c-a6e300b2acf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.691262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fc9cf-2787-426a-958c-a6e300b2acf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.691980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fc9cf-2787-426a-958c-a6e300b2acf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:17.693607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fc9cf-2787-426a-958c-a6e300b2acf4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:17.704226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1049/2114] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:17.708910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff63fc8-3b25-45d4-8dd7-20c48fd9aca1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:17.719261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff63fc8-3b25-45d4-8dd7-20c48fd9aca1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:17.724339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfd06d-0762-4e45-ac18-16bc00441716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:17.725257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfd06d-0762-4e45-ac18-16bc00441716 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.725533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfd06d-0762-4e45-ac18-16bc00441716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.725805 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfd06d-0762-4e45-ac18-16bc00441716 admin admin] PATH_INFO: `/v3/projects/75ee619f4c5547e8b107ad3e9e36d6af/users/733e9600bb7c4b5c8426c90133d6e958/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.741750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfd06d-0762-4e45-ac18-16bc00441716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ee619f4c5547e8b107ad3e9e36d6af, user_id=733e9600bb7c4b5c8426c90133d6e958, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:17.744293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfd06d-0762-4e45-ac18-16bc00441716 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:17.759598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1066/2115] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:17 2022] PUT /identity/v3/projects/75ee619f4c5547e8b107ad3e9e36d6af/users/733e9600bb7c4b5c8426c90133d6e958/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:17.771958 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd0625-6d48-48f1-ba94-e7aef51f9ddc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.772310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd0625-6d48-48f1-ba94-e7aef51f9ddc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.772644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd0625-6d48-48f1-ba94-e7aef51f9ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.802793 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7dbd0625-6d48-48f1-ba94-e7aef51f9ddc None None] MFA Rules not processed for user `733e9600bb7c4b5c8426c90133d6e958`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:17.804026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7dbd0625-6d48-48f1-ba94-e7aef51f9ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:17.870989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1050/2116] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:17.875646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfd06d-0762-4e45-ac18-16bc00441716 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:17.951733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfd06d-0762-4e45-ac18-16bc00441716 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:17.957762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231de03-d996-4275-8b0e-3d17d02c0602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:17.958514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231de03-d996-4275-8b0e-3d17d02c0602 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:17.958794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231de03-d996-4275-8b0e-3d17d02c0602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:17.959068 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231de03-d996-4275-8b0e-3d17d02c0602 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:17.959762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231de03-d996-4275-8b0e-3d17d02c0602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:17.961750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231de03-d996-4275-8b0e-3d17d02c0602 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:17.990583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1067/2117] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:17 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:17.995077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd0625-6d48-48f1-ba94-e7aef51f9ddc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:18.006989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd0625-6d48-48f1-ba94-e7aef51f9ddc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:18.063942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c25fe7-2e1b-47ec-a677-699716369aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:18.065559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c25fe7-2e1b-47ec-a677-699716369aca admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:18.065875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c25fe7-2e1b-47ec-a677-699716369aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:18.066150 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c25fe7-2e1b-47ec-a677-699716369aca admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:18.067092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c25fe7-2e1b-47ec-a677-699716369aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:18.069254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c25fe7-2e1b-47ec-a677-699716369aca admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:18.092815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1051/2118] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:17 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:18.097118 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231de03-d996-4275-8b0e-3d17d02c0602 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:18.108066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231de03-d996-4275-8b0e-3d17d02c0602 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:18.119752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e1bda-804a-45d9-b885-f372236910ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:18.119752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e1bda-804a-45d9-b885-f372236910ab admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:18.119752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e1bda-804a-45d9-b885-f372236910ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:18.119752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e1bda-804a-45d9-b885-f372236910ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:18.120115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e1bda-804a-45d9-b885-f372236910ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:18.120115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e1bda-804a-45d9-b885-f372236910ab admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:18.127380 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1068/2119] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:18.131960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c25fe7-2e1b-47ec-a677-699716369aca admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:18.142707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c25fe7-2e1b-47ec-a677-699716369aca admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:18.147722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe75ff-99ec-46d2-af2b-6a97c8ac6ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:18.148759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe75ff-99ec-46d2-af2b-6a97c8ac6ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:18.149032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe75ff-99ec-46d2-af2b-6a97c8ac6ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:18.149302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe75ff-99ec-46d2-af2b-6a97c8ac6ccc admin admin] PATH_INFO: `/v3/projects/eae40e82055d4901861b85f43934ee9d/users/b0cd58f2696f4155932ff603101e671b/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:18.163787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe75ff-99ec-46d2-af2b-6a97c8ac6ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae40e82055d4901861b85f43934ee9d, user_id=b0cd58f2696f4155932ff603101e671b, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:18.166473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe75ff-99ec-46d2-af2b-6a97c8ac6ccc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:18.180534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1052/2120] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:18 2022] PUT /identity/v3/projects/eae40e82055d4901861b85f43934ee9d/users/b0cd58f2696f4155932ff603101e671b/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:18.184970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e1bda-804a-45d9-b885-f372236910ab admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:18.196126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e1bda-804a-45d9-b885-f372236910ab admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:18.269327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c0092-ceb9-47fb-8122-d53e31a95e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:18.269917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c0092-ceb9-47fb-8122-d53e31a95e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:18.270199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c0092-ceb9-47fb-8122-d53e31a95e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:18.270469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c0092-ceb9-47fb-8122-d53e31a95e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:18.271216 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c0092-ceb9-47fb-8122-d53e31a95e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:18.272867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c0092-ceb9-47fb-8122-d53e31a95e56 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:18.284200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1069/2121] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:18.288751 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe75ff-99ec-46d2-af2b-6a97c8ac6ccc admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:18.300057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe75ff-99ec-46d2-af2b-6a97c8ac6ccc admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:18.306294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354e755-b301-440a-922d-86f82eb6694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:18.307221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354e755-b301-440a-922d-86f82eb6694d admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:18.307486 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354e755-b301-440a-922d-86f82eb6694d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:18.307760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354e755-b301-440a-922d-86f82eb6694d admin admin] PATH_INFO: `/v3/projects/eae40e82055d4901861b85f43934ee9d/users/b0cd58f2696f4155932ff603101e671b/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:18.310023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354e755-b301-440a-922d-86f82eb6694d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae40e82055d4901861b85f43934ee9d, user_id=b0cd58f2696f4155932ff603101e671b, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:18.312777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354e755-b301-440a-922d-86f82eb6694d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:18.324739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1053/2122] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:18 2022] PUT /identity/v3/projects/eae40e82055d4901861b85f43934ee9d/users/b0cd58f2696f4155932ff603101e671b/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:18.336308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc358c4-0a47-4338-a022-4f1b98346bc2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:18.336629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc358c4-0a47-4338-a022-4f1b98346bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:18.336921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc358c4-0a47-4338-a022-4f1b98346bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:18.367198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4bc358c4-0a47-4338-a022-4f1b98346bc2 None None] MFA Rules not processed for user `b0cd58f2696f4155932ff603101e671b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:18.368418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4bc358c4-0a47-4338-a022-4f1b98346bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:18.438554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1070/2123] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:18.450190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354e755-b301-440a-922d-86f82eb6694d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:18.452601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f354e755-b301-440a-922d-86f82eb6694d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:18.511956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354e755-b301-440a-922d-86f82eb6694d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:18.515960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bd47d-b8a2-4acc-9203-107445c64b9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:18.516990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd47d-b8a2-4acc-9203-107445c64b9b service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:18.516990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd47d-b8a2-4acc-9203-107445c64b9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:18.517321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd47d-b8a2-4acc-9203-107445c64b9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:18.527960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bd47d-b8a2-4acc-9203-107445c64b9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:18.529551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bd47d-b8a2-4acc-9203-107445c64b9b service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:18.540130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1054/2124] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:18.909244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc358c4-0a47-4338-a022-4f1b98346bc2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:18.970708 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc358c4-0a47-4338-a022-4f1b98346bc2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:18.978667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:18.980103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:18.980578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:18.981082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] PATH_INFO: `/v3/users/733e9600bb7c4b5c8426c90133d6e958` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:18.983319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=733e9600bb7c4b5c8426c90133d6e958)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:18.986563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:19.039434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733e9600bb7c4b5c8426c90133d6e958'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.040175 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733e9600bb7c4b5c8426c90133d6e958'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.053729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733e9600bb7c4b5c8426c90133d6e958'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.063526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733e9600bb7c4b5c8426c90133d6e958'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.070906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733e9600bb7c4b5c8426c90133d6e958'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.077405 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1071/2125] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:18 2022] DELETE /identity/v3/users/733e9600bb7c4b5c8426c90133d6e958 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:19.082235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c303ea9-c4c1-4dfa-bb7d-710570d78599 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:19.082731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c303ea9-c4c1-4dfa-bb7d-710570d78599 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:19.082822 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c303ea9-c4c1-4dfa-bb7d-710570d78599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:19.100672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7c303ea9-c4c1-4dfa-bb7d-710570d78599 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:19.101865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7c303ea9-c4c1-4dfa-bb7d-710570d78599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:19.163716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1055/2126] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:19.175648 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:19.177133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:19.239514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617f509-5d4d-4ac8-87c2-b74da058b535 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:19.245732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b648-a4c0-4a11-b7bc-3c613210e762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:19.246538 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b648-a4c0-4a11-b7bc-3c613210e762 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:19.246945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b648-a4c0-4a11-b7bc-3c613210e762 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:19.247302 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b648-a4c0-4a11-b7bc-3c613210e762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:19.258622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b648-a4c0-4a11-b7bc-3c613210e762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:19.260050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b648-a4c0-4a11-b7bc-3c613210e762 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:19.269573 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1072/2127] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:19.548429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c303ea9-c4c1-4dfa-bb7d-710570d78599 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:19.550585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7c303ea9-c4c1-4dfa-bb7d-710570d78599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:19.562825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c303ea9-c4c1-4dfa-bb7d-710570d78599 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:19.568180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:19.569138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:19.569421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:19.570211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] PATH_INFO: `/v3/projects/75ee619f4c5547e8b107ad3e9e36d6af` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:19.571660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ee619f4c5547e8b107ad3e9e36d6af)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:19.573723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:19.626624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ee619f4c5547e8b107ad3e9e36d6af', 'request_id': 'req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.636031 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ee619f4c5547e8b107ad3e9e36d6af', 'request_id': 'req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.643109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ee619f4c5547e8b107ad3e9e36d6af', 'request_id': 'req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.653085 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] The token cache is being invalidate because project 75ee619f4c5547e8b107ad3e9e36d6af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:19.653365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.654021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.655579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1056/2128] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:19 2022] DELETE /identity/v3/projects/75ee619f4c5547e8b107ad3e9e36d6af => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:19.659747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b648-a4c0-4a11-b7bc-3c613210e762 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:19.669870 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b648-a4c0-4a11-b7bc-3c613210e762 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:19.731916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:19.732872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:19.733157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:19.733426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] PATH_INFO: `/v3/users/b0cd58f2696f4155932ff603101e671b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:19.734774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0cd58f2696f4155932ff603101e671b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:19.737019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:19.785826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0cd58f2696f4155932ff603101e671b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.786551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0cd58f2696f4155932ff603101e671b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.803510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0cd58f2696f4155932ff603101e671b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.813046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0cd58f2696f4155932ff603101e671b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.822411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b4ceeb8c-d2b2-452e-aae0-55e25bee54fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0cd58f2696f4155932ff603101e671b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:19.832720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1073/2129] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:19 2022] DELETE /identity/v3/users/b0cd58f2696f4155932ff603101e671b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:20.106593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:20.108574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:20.125938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9005b9-bc6b-404d-ba60-4354bbe0f18d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:20.189578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:20.191193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.191474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.191748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] PATH_INFO: `/v3/projects/eae40e82055d4901861b85f43934ee9d` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.193097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae40e82055d4901861b85f43934ee9d)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:20.195039 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:20.244293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae40e82055d4901861b85f43934ee9d', 'request_id': 'req-7f6a7d38-3841-46ee-aec9-682410d50521'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:20.252830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae40e82055d4901861b85f43934ee9d', 'request_id': 'req-7f6a7d38-3841-46ee-aec9-682410d50521'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:20.259329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae40e82055d4901861b85f43934ee9d', 'request_id': 'req-7f6a7d38-3841-46ee-aec9-682410d50521'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:20.270523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] The token cache is being invalidate because project eae40e82055d4901861b85f43934ee9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:20.270807 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:20.271552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7f6a7d38-3841-46ee-aec9-682410d50521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:20.273393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1057/2130] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:20 2022] DELETE /identity/v3/projects/eae40e82055d4901861b85f43934ee9d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:20.278816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d6d7b-281a-4260-aa6a-3ee1c5172d9d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.279398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d6d7b-281a-4260-aa6a-3ee1c5172d9d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.279398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d6d7b-281a-4260-aa6a-3ee1c5172d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.298391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f78d6d7b-281a-4260-aa6a-3ee1c5172d9d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:20.303222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f78d6d7b-281a-4260-aa6a-3ee1c5172d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:20.368901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1074/2131] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:20.378239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803f2ac-89f4-41c2-babd-7f4288d5339a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.378529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803f2ac-89f4-41c2-babd-7f4288d5339a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.378806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803f2ac-89f4-41c2-babd-7f4288d5339a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.400067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9803f2ac-89f4-41c2-babd-7f4288d5339a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:20.401332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9803f2ac-89f4-41c2-babd-7f4288d5339a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:20.411373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1058/2132] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:20.415961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d6d7b-281a-4260-aa6a-3ee1c5172d9d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:20.426756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d6d7b-281a-4260-aa6a-3ee1c5172d9d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:20.431568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e8196-f497-4e27-838d-80be8e3d0fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:20.432239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e8196-f497-4e27-838d-80be8e3d0fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.432528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e8196-f497-4e27-838d-80be8e3d0fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.432812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e8196-f497-4e27-838d-80be8e3d0fa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.433515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e8196-f497-4e27-838d-80be8e3d0fa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:45:20.433812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e8196-f497-4e27-838d-80be8e3d0fa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:20.435459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e8196-f497-4e27-838d-80be8e3d0fa0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:20.446710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1075/2133] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:45:20.451878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803f2ac-89f4-41c2-babd-7f4288d5339a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:20.462123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803f2ac-89f4-41c2-babd-7f4288d5339a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:20.468073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00f40b-b449-4b63-aef5-99230dd43ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:20.468842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00f40b-b449-4b63-aef5-99230dd43ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.469150 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00f40b-b449-4b63-aef5-99230dd43ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.469422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00f40b-b449-4b63-aef5-99230dd43ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.470292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00f40b-b449-4b63-aef5-99230dd43ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:20.472763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00f40b-b449-4b63-aef5-99230dd43ca8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:20.502159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1059/2134] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:20 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:20.506497 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e8196-f497-4e27-838d-80be8e3d0fa0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:20.516876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e8196-f497-4e27-838d-80be8e3d0fa0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:20.573760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8d97d-5303-4519-896c-b4eed6d622e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:20.574423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8d97d-5303-4519-896c-b4eed6d622e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.574697 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8d97d-5303-4519-896c-b4eed6d622e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.574968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8d97d-5303-4519-896c-b4eed6d622e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.576010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8d97d-5303-4519-896c-b4eed6d622e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:20.577982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8d97d-5303-4519-896c-b4eed6d622e9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:20.602706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1076/2135] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:20 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:20.607689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00f40b-b449-4b63-aef5-99230dd43ca8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:20.619268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00f40b-b449-4b63-aef5-99230dd43ca8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:20.624859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df1108-35e8-4b3b-bcae-b6e0a4548d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:20.625525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df1108-35e8-4b3b-bcae-b6e0a4548d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.625800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df1108-35e8-4b3b-bcae-b6e0a4548d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.626065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df1108-35e8-4b3b-bcae-b6e0a4548d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.626789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df1108-35e8-4b3b-bcae-b6e0a4548d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:20.628567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df1108-35e8-4b3b-bcae-b6e0a4548d84 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:20.639327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1060/2136] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:20.643889 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8d97d-5303-4519-896c-b4eed6d622e9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:20.654874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8d97d-5303-4519-896c-b4eed6d622e9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:20.661135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba122819-af65-48dd-88a7-9365c8e5c403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:20.662131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba122819-af65-48dd-88a7-9365c8e5c403 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.662512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba122819-af65-48dd-88a7-9365c8e5c403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.662800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba122819-af65-48dd-88a7-9365c8e5c403 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.663843 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba122819-af65-48dd-88a7-9365c8e5c403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:20.666051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba122819-af65-48dd-88a7-9365c8e5c403 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:20.678000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1077/2137] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:20.682957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df1108-35e8-4b3b-bcae-b6e0a4548d84 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:20.694479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df1108-35e8-4b3b-bcae-b6e0a4548d84 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:20.699415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73df7a-d5a2-4d78-bf48-9bd960afd8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:20.700310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73df7a-d5a2-4d78-bf48-9bd960afd8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.701107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73df7a-d5a2-4d78-bf48-9bd960afd8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.701349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73df7a-d5a2-4d78-bf48-9bd960afd8f1 admin admin] PATH_INFO: `/v3/projects/a64a94182d7d46ee8359295b763feff8/users/7fddc6ba983b45998e9628f8c90263aa/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.718520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73df7a-d5a2-4d78-bf48-9bd960afd8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a64a94182d7d46ee8359295b763feff8, user_id=7fddc6ba983b45998e9628f8c90263aa, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:20.721442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73df7a-d5a2-4d78-bf48-9bd960afd8f1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:20.736814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1061/2138] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:20 2022] PUT /identity/v3/projects/a64a94182d7d46ee8359295b763feff8/users/7fddc6ba983b45998e9628f8c90263aa/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:20.747850 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cab70-623e-4f2b-b4fc-f37d26f0cf17 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.748137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cab70-623e-4f2b-b4fc-f37d26f0cf17 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.748413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cab70-623e-4f2b-b4fc-f37d26f0cf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.774810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-414cab70-623e-4f2b-b4fc-f37d26f0cf17 None None] MFA Rules not processed for user `7fddc6ba983b45998e9628f8c90263aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:20.775987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-414cab70-623e-4f2b-b4fc-f37d26f0cf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:20.848326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1078/2139] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:20.859582 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73df7a-d5a2-4d78-bf48-9bd960afd8f1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:20.861828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ea73df7a-d5a2-4d78-bf48-9bd960afd8f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:20.922333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73df7a-d5a2-4d78-bf48-9bd960afd8f1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:20.926454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b59cd0-1326-4023-92b6-f700f2aae9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:20.927207 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59cd0-1326-4023-92b6-f700f2aae9ed service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:20.927514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59cd0-1326-4023-92b6-f700f2aae9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:20.927776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59cd0-1326-4023-92b6-f700f2aae9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:20.936774 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b59cd0-1326-4023-92b6-f700f2aae9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:20.938161 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b59cd0-1326-4023-92b6-f700f2aae9ed service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:20.947326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1062/2140] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:41.840069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cab70-623e-4f2b-b4fc-f37d26f0cf17 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:41.950761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cab70-623e-4f2b-b4fc-f37d26f0cf17 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:41.955388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:41.956259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:41.956555 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:41.956828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] PATH_INFO: `/v3/users/7fddc6ba983b45998e9628f8c90263aa` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:41.958054 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fddc6ba983b45998e9628f8c90263aa)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:41.959946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:42.006211 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fddc6ba983b45998e9628f8c90263aa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.006771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fddc6ba983b45998e9628f8c90263aa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.017645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fddc6ba983b45998e9628f8c90263aa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.026401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fddc6ba983b45998e9628f8c90263aa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.034670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fddc6ba983b45998e9628f8c90263aa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.041349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1079/2141] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:41 2022] DELETE /identity/v3/users/7fddc6ba983b45998e9628f8c90263aa => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:42.046966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c885c-0b23-45df-b689-8215fcb8fa96 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:42.047468 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c885c-0b23-45df-b689-8215fcb8fa96 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:42.047563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c885c-0b23-45df-b689-8215fcb8fa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:42.097686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-2f0c885c-0b23-45df-b689-8215fcb8fa96 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:42.098873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2f0c885c-0b23-45df-b689-8215fcb8fa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:42.156903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1063/2142] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:42.166595 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:42.168069 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:42.262518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567ad49-5e73-4a9a-bd73-43368de5eccb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:42.266532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b6043-4b8e-4e4c-ab6a-640d9b66b086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:42.267174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b6043-4b8e-4e4c-ab6a-640d9b66b086 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:42.267459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b6043-4b8e-4e4c-ab6a-640d9b66b086 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:42.267723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b6043-4b8e-4e4c-ab6a-640d9b66b086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:42.276615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b6043-4b8e-4e4c-ab6a-640d9b66b086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:42.278035 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b6043-4b8e-4e4c-ab6a-640d9b66b086 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:42.287231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1080/2143] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:42.564975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c885c-0b23-45df-b689-8215fcb8fa96 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:42.566522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2f0c885c-0b23-45df-b689-8215fcb8fa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:42.580248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c885c-0b23-45df-b689-8215fcb8fa96 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:42.586195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:42.587086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:42.587370 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:42.587738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] PATH_INFO: `/v3/projects/a64a94182d7d46ee8359295b763feff8` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:42.591609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a64a94182d7d46ee8359295b763feff8)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:42.594003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:42.646619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a64a94182d7d46ee8359295b763feff8', 'request_id': 'req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.654151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a64a94182d7d46ee8359295b763feff8', 'request_id': 'req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.662483 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a64a94182d7d46ee8359295b763feff8', 'request_id': 'req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.674419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] The token cache is being invalidate because project a64a94182d7d46ee8359295b763feff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:42.674706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.675361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e2ba26d8-e5d9-4c7b-98df-59f2c8597917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.677028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1064/2144] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:42 2022] DELETE /identity/v3/projects/a64a94182d7d46ee8359295b763feff8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:42.682128 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9c308-827d-4b2c-b49d-70d5953ac644 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:42.682417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9c308-827d-4b2c-b49d-70d5953ac644 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:42.682690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9c308-827d-4b2c-b49d-70d5953ac644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:42.722610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-63f9c308-827d-4b2c-b49d-70d5953ac644 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:42.723823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-63f9c308-827d-4b2c-b49d-70d5953ac644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:42.786333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1081/2145] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:42.794315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a5b12-d507-4788-b7e4-cff6aa23ebd6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:42.794609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a5b12-d507-4788-b7e4-cff6aa23ebd6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:42.794886 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a5b12-d507-4788-b7e4-cff6aa23ebd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:42.814415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f28a5b12-d507-4788-b7e4-cff6aa23ebd6 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:42.815625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f28a5b12-d507-4788-b7e4-cff6aa23ebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:42.824971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1065/2146] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:42.829922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9c308-827d-4b2c-b49d-70d5953ac644 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:42.840401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9c308-827d-4b2c-b49d-70d5953ac644 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:42.848769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af7b6f-5e8c-4d58-af8c-eb23fffe16f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:42.849736 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7b6f-5e8c-4d58-af8c-eb23fffe16f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:42.850017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7b6f-5e8c-4d58-af8c-eb23fffe16f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:42.850311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7b6f-5e8c-4d58-af8c-eb23fffe16f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:42.851352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af7b6f-5e8c-4d58-af8c-eb23fffe16f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:45:42.851715 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af7b6f-5e8c-4d58-af8c-eb23fffe16f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:42.854049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af7b6f-5e8c-4d58-af8c-eb23fffe16f5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:42.867242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1082/2147] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:45:42.871728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a5b12-d507-4788-b7e4-cff6aa23ebd6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:42.881455 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a5b12-d507-4788-b7e4-cff6aa23ebd6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:42.887416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2a117-9a19-4658-ab94-736bcbaee72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:42.888155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2a117-9a19-4658-ab94-736bcbaee72c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:42.888530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2a117-9a19-4658-ab94-736bcbaee72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:42.889172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2a117-9a19-4658-ab94-736bcbaee72c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:42.890040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2a117-9a19-4658-ab94-736bcbaee72c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:42.895608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2a117-9a19-4658-ab94-736bcbaee72c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:42.901407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af7b6f-5e8c-4d58-af8c-eb23fffe16f5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:42.903527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-82af7b6f-5e8c-4d58-af8c-eb23fffe16f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:42.918852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af7b6f-5e8c-4d58-af8c-eb23fffe16f5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:42.923133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1066/2148] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:42 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:42.924134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:42.925037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:42.925318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:42.925587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] PATH_INFO: `/v3/users/94185fb7a64d4d8da8b1ce4ba30bc37f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:42.927137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94185fb7a64d4d8da8b1ce4ba30bc37f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:42.927270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2a117-9a19-4658-ab94-736bcbaee72c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:42.929101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:42.978848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94185fb7a64d4d8da8b1ce4ba30bc37f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.979522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94185fb7a64d4d8da8b1ce4ba30bc37f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.991630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94185fb7a64d4d8da8b1ce4ba30bc37f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:42.999337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94185fb7a64d4d8da8b1ce4ba30bc37f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:43.000067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2a117-9a19-4658-ab94-736bcbaee72c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:43.004631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92f961-6b26-45c2-ac2f-e509538b61e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:43.005301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92f961-6b26-45c2-ac2f-e509538b61e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.005578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92f961-6b26-45c2-ac2f-e509538b61e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.005866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92f961-6b26-45c2-ac2f-e509538b61e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.006388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c491352a-6d85-49f0-8a47-c3c23a356f17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94185fb7a64d4d8da8b1ce4ba30bc37f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:43.006549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92f961-6b26-45c2-ac2f-e509538b61e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:43.008446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92f961-6b26-45c2-ac2f-e509538b61e4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:43.012322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1083/2149] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:42 2022] DELETE /identity/v3/users/94185fb7a64d4d8da8b1ce4ba30bc37f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:43.016713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5eff53-2051-4295-8b8a-65d3d9d5befe None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.017001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5eff53-2051-4295-8b8a-65d3d9d5befe None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.017273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5eff53-2051-4295-8b8a-65d3d9d5befe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.031172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1067/2150] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:42 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:45:43.032203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92f961-6b26-45c2-ac2f-e509538b61e4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:43.033812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ec92f961-6b26-45c2-ac2f-e509538b61e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:43.035256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2d5eff53-2051-4295-8b8a-65d3d9d5befe None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:43.036463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2d5eff53-2051-4295-8b8a-65d3d9d5befe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:43.045737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92f961-6b26-45c2-ac2f-e509538b61e4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:43.093356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1084/2151] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:45:43.099535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12a0f4-b544-4b85-ac12-4244bd08e676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:43.100211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12a0f4-b544-4b85-ac12-4244bd08e676 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.100520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12a0f4-b544-4b85-ac12-4244bd08e676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.100792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12a0f4-b544-4b85-ac12-4244bd08e676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.101512 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12a0f4-b544-4b85-ac12-4244bd08e676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:43.103186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12a0f4-b544-4b85-ac12-4244bd08e676 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:43.103366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5eff53-2051-4295-8b8a-65d3d9d5befe None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:43.104790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2d5eff53-2051-4295-8b8a-65d3d9d5befe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:43.113510 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1068/2152] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:45:43.117179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12a0f4-b544-4b85-ac12-4244bd08e676 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:43.126616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12a0f4-b544-4b85-ac12-4244bd08e676 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:43.131317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5448f-b040-49d4-9ce8-8d408e236a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:43.132023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5448f-b040-49d4-9ce8-8d408e236a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.132303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5448f-b040-49d4-9ce8-8d408e236a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.132615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5448f-b040-49d4-9ce8-8d408e236a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.133335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5448f-b040-49d4-9ce8-8d408e236a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:43.135088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5448f-b040-49d4-9ce8-8d408e236a82 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:43.145145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1069/2153] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:45:43.149124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5448f-b040-49d4-9ce8-8d408e236a82 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:43.158780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5448f-b040-49d4-9ce8-8d408e236a82 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:43.163587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5eff53-2051-4295-8b8a-65d3d9d5befe None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:43.164142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79732594-8764-421e-aa97-7bf81b010f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:43.165264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79732594-8764-421e-aa97-7bf81b010f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.165701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79732594-8764-421e-aa97-7bf81b010f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.166114 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79732594-8764-421e-aa97-7bf81b010f1c admin admin] PATH_INFO: `/v3/projects/131b4943a4aa42a8b09d614d80f552cc/users/0ad000d2e49f4b6fa0e9315d35e7d4da/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.169589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c47300-55de-479b-b699-ab928d65efba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:43.170300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c47300-55de-479b-b699-ab928d65efba service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.170578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c47300-55de-479b-b699-ab928d65efba service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.170892 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c47300-55de-479b-b699-ab928d65efba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.180324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c47300-55de-479b-b699-ab928d65efba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:43.181794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c47300-55de-479b-b699-ab928d65efba service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:43.182920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79732594-8764-421e-aa97-7bf81b010f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=131b4943a4aa42a8b09d614d80f552cc, user_id=0ad000d2e49f4b6fa0e9315d35e7d4da, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:43.185593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79732594-8764-421e-aa97-7bf81b010f1c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:43.191265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1085/2154] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:45:43.198977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1070/2155] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:43 2022] PUT /identity/v3/projects/131b4943a4aa42a8b09d614d80f552cc/users/0ad000d2e49f4b6fa0e9315d35e7d4da/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:43.204410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0d64c-7799-41f6-8bc4-1de5c0657fa9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.204735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0d64c-7799-41f6-8bc4-1de5c0657fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.205010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0d64c-7799-41f6-8bc4-1de5c0657fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.222107 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-66b0d64c-7799-41f6-8bc4-1de5c0657fa9 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:43.223371 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-66b0d64c-7799-41f6-8bc4-1de5c0657fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:43.288122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1086/2156] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:43.298895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79732594-8764-421e-aa97-7bf81b010f1c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:43.300857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-79732594-8764-421e-aa97-7bf81b010f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:43.359984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79732594-8764-421e-aa97-7bf81b010f1c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:43.363691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba764c-00db-438f-96e2-213c58b58555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:43.364343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba764c-00db-438f-96e2-213c58b58555 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.364652 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba764c-00db-438f-96e2-213c58b58555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.364912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba764c-00db-438f-96e2-213c58b58555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.374487 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba764c-00db-438f-96e2-213c58b58555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:43.376010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba764c-00db-438f-96e2-213c58b58555 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:43.388596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1071/2157] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:43.581727 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0d64c-7799-41f6-8bc4-1de5c0657fa9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:43.583253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-66b0d64c-7799-41f6-8bc4-1de5c0657fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:43.596343 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0d64c-7799-41f6-8bc4-1de5c0657fa9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:43.614109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:43.615102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.615372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.615680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] PATH_INFO: `/v3/projects/bca9182d4e9d49e89d4e6a163c21eba5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.617163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca9182d4e9d49e89d4e6a163c21eba5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:43.619137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:43.668592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca9182d4e9d49e89d4e6a163c21eba5', 'request_id': 'req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:43.677785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca9182d4e9d49e89d4e6a163c21eba5', 'request_id': 'req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:43.684208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca9182d4e9d49e89d4e6a163c21eba5', 'request_id': 'req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:43.695976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] The token cache is being invalidate because project bca9182d4e9d49e89d4e6a163c21eba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:43.696310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:43.697010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc28aeb5-d04d-4c7d-9951-0eea5da332ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:43.698572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1087/2158] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:43 2022] DELETE /identity/v3/projects/bca9182d4e9d49e89d4e6a163c21eba5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:43.704471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d410f-cf3a-4b1a-b035-ec97a852360d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.704648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d410f-cf3a-4b1a-b035-ec97a852360d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.704851 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d410f-cf3a-4b1a-b035-ec97a852360d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.735528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-614d410f-cf3a-4b1a-b035-ec97a852360d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:43.736752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-614d410f-cf3a-4b1a-b035-ec97a852360d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:43.808725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1072/2159] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:43.820830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0db98-98c4-4d36-8cf9-d23b3f5da5c8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.821262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0db98-98c4-4d36-8cf9-d23b3f5da5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.821587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0db98-98c4-4d36-8cf9-d23b3f5da5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.845682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b9c0db98-98c4-4d36-8cf9-d23b3f5da5c8 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:43.848956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b9c0db98-98c4-4d36-8cf9-d23b3f5da5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:43.858772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1088/2160] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:43.863147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d410f-cf3a-4b1a-b035-ec97a852360d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:43.874887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d410f-cf3a-4b1a-b035-ec97a852360d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:43.880846 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1ba37-bf5a-411b-b8c2-2ce4e4b004f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:43.881688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1ba37-bf5a-411b-b8c2-2ce4e4b004f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.882034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1ba37-bf5a-411b-b8c2-2ce4e4b004f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.882381 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1ba37-bf5a-411b-b8c2-2ce4e4b004f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.883196 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1ba37-bf5a-411b-b8c2-2ce4e4b004f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:45:43.883577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1ba37-bf5a-411b-b8c2-2ce4e4b004f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:43.885304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1ba37-bf5a-411b-b8c2-2ce4e4b004f1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:43.900997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1073/2161] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:45:43.902461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0db98-98c4-4d36-8cf9-d23b3f5da5c8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:43.913895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0db98-98c4-4d36-8cf9-d23b3f5da5c8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:43.919690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deecfea-9da3-4698-ae99-e7a41e48db3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:43.920684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deecfea-9da3-4698-ae99-e7a41e48db3a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:43.921032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deecfea-9da3-4698-ae99-e7a41e48db3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:43.921354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deecfea-9da3-4698-ae99-e7a41e48db3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:43.922117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deecfea-9da3-4698-ae99-e7a41e48db3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:43.924840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deecfea-9da3-4698-ae99-e7a41e48db3a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:43.955782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1089/2162] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:43 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:43.959734 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1ba37-bf5a-411b-b8c2-2ce4e4b004f1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:43.970771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1ba37-bf5a-411b-b8c2-2ce4e4b004f1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:44.050017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1d1fd-1581-445a-b844-305f281511a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:44.050698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1d1fd-1581-445a-b844-305f281511a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:44.050981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1d1fd-1581-445a-b844-305f281511a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:44.051249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1d1fd-1581-445a-b844-305f281511a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:44.052066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1d1fd-1581-445a-b844-305f281511a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:44.054059 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1d1fd-1581-445a-b844-305f281511a7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:44.075835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1074/2163] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:43 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:44.080160 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deecfea-9da3-4698-ae99-e7a41e48db3a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:44.091241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deecfea-9da3-4698-ae99-e7a41e48db3a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:44.098327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78918dd-30e4-4062-b8e9-e590d92fb20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:44.099032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78918dd-30e4-4062-b8e9-e590d92fb20c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:44.099536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78918dd-30e4-4062-b8e9-e590d92fb20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:44.099809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78918dd-30e4-4062-b8e9-e590d92fb20c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:44.100878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78918dd-30e4-4062-b8e9-e590d92fb20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:44.102995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78918dd-30e4-4062-b8e9-e590d92fb20c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:44.117422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1090/2164] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:44.121453 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1d1fd-1581-445a-b844-305f281511a7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:44.132295 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1d1fd-1581-445a-b844-305f281511a7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:44.137033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35672ef-6255-40a5-a346-eae8bc99b579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:44.137747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35672ef-6255-40a5-a346-eae8bc99b579 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:44.137968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35672ef-6255-40a5-a346-eae8bc99b579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:44.138238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35672ef-6255-40a5-a346-eae8bc99b579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:44.139129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35672ef-6255-40a5-a346-eae8bc99b579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:44.140816 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35672ef-6255-40a5-a346-eae8bc99b579 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:44.150937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1075/2165] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:44.155185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78918dd-30e4-4062-b8e9-e590d92fb20c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:44.164839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78918dd-30e4-4062-b8e9-e590d92fb20c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:44.170329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b5584-db3f-4700-8bfc-6fc0853fbd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:44.171231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b5584-db3f-4700-8bfc-6fc0853fbd1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:44.171511 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b5584-db3f-4700-8bfc-6fc0853fbd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:44.171780 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b5584-db3f-4700-8bfc-6fc0853fbd1f admin admin] PATH_INFO: `/v3/projects/941e38a2e73a4905ac8568f1af5af294/users/06e1c95f942c4ce0b8487b62b3e293fb/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:44.187550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b5584-db3f-4700-8bfc-6fc0853fbd1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=941e38a2e73a4905ac8568f1af5af294, user_id=06e1c95f942c4ce0b8487b62b3e293fb, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:44.190190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b5584-db3f-4700-8bfc-6fc0853fbd1f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:44.219497 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1091/2166] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:44 2022] PUT /identity/v3/projects/941e38a2e73a4905ac8568f1af5af294/users/06e1c95f942c4ce0b8487b62b3e293fb/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:44.224787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7881fd-2ef1-4b54-9bcd-5dc7300cd0b9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:44.225083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7881fd-2ef1-4b54-9bcd-5dc7300cd0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:44.225402 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7881fd-2ef1-4b54-9bcd-5dc7300cd0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:44.253073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1b7881fd-2ef1-4b54-9bcd-5dc7300cd0b9 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:44.254517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1b7881fd-2ef1-4b54-9bcd-5dc7300cd0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:44.317890 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1076/2167] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:44.331712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b5584-db3f-4700-8bfc-6fc0853fbd1f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:44.333532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f71b5584-db3f-4700-8bfc-6fc0853fbd1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:44.399273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b5584-db3f-4700-8bfc-6fc0853fbd1f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:44.406592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958b10a-2d48-421d-8bd8-6552b76ca9d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:44.406592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958b10a-2d48-421d-8bd8-6552b76ca9d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:44.406592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958b10a-2d48-421d-8bd8-6552b76ca9d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:44.407103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958b10a-2d48-421d-8bd8-6552b76ca9d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:44.417588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958b10a-2d48-421d-8bd8-6552b76ca9d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:44.419046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958b10a-2d48-421d-8bd8-6552b76ca9d7 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:44.429237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1092/2168] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:45.807609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a857e-2fdd-426c-beb0-5dfc186fa5cd None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:45.808182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a857e-2fdd-426c-beb0-5dfc186fa5cd None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:45.808182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a857e-2fdd-426c-beb0-5dfc186fa5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:45.838733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e07a857e-2fdd-426c-beb0-5dfc186fa5cd None None] MFA Rules not processed for user `06e1c95f942c4ce0b8487b62b3e293fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:45.840209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e07a857e-2fdd-426c-beb0-5dfc186fa5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:45.943229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1077/2169] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:45.956224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958b10a-2d48-421d-8bd8-6552b76ca9d7 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:45.962556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c958b10a-2d48-421d-8bd8-6552b76ca9d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:46.037814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958b10a-2d48-421d-8bd8-6552b76ca9d7 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:46.044325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c2b6d-e883-4c11-b1eb-8f772e0f65e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:46.045266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c2b6d-e883-4c11-b1eb-8f772e0f65e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:46.045615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c2b6d-e883-4c11-b1eb-8f772e0f65e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:46.046083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c2b6d-e883-4c11-b1eb-8f772e0f65e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:46.057507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c2b6d-e883-4c11-b1eb-8f772e0f65e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:46.059681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c2b6d-e883-4c11-b1eb-8f772e0f65e5 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:46.071925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1093/2170] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:48.688642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc158d6-8c2f-4e6b-a117-1ea79e4be401 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:48.689168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc158d6-8c2f-4e6b-a117-1ea79e4be401 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:48.689168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc158d6-8c2f-4e6b-a117-1ea79e4be401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:48.715482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ccc158d6-8c2f-4e6b-a117-1ea79e4be401 None None] MFA Rules not processed for user `06e1c95f942c4ce0b8487b62b3e293fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:48.716747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ccc158d6-8c2f-4e6b-a117-1ea79e4be401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:48.741746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1078/2171] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:49.534968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1127-41ce-4c5d-a1bb-3025eef94088 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:49.535475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1127-41ce-4c5d-a1bb-3025eef94088 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:49.535553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1127-41ce-4c5d-a1bb-3025eef94088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:49.571092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-810f1127-41ce-4c5d-a1bb-3025eef94088 None None] MFA Rules not processed for user `0ad000d2e49f4b6fa0e9315d35e7d4da`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:49.572399 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-810f1127-41ce-4c5d-a1bb-3025eef94088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:49.644674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1094/2172] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:49.649242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc158d6-8c2f-4e6b-a117-1ea79e4be401 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:49.650761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ccc158d6-8c2f-4e6b-a117-1ea79e4be401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:49.664730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc158d6-8c2f-4e6b-a117-1ea79e4be401 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:49.669367 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd859-bc39-435b-ac46-ba9bb93251ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:49.670158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd859-bc39-435b-ac46-ba9bb93251ff admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:49.670434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd859-bc39-435b-ac46-ba9bb93251ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:49.670707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd859-bc39-435b-ac46-ba9bb93251ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:49.671404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfd859-bc39-435b-ac46-ba9bb93251ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:49.673361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfd859-bc39-435b-ac46-ba9bb93251ff admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:49.709781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1079/2173] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:49 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:49.714022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1127-41ce-4c5d-a1bb-3025eef94088 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:49.727233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1127-41ce-4c5d-a1bb-3025eef94088 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:49.801619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa323d6-4093-4da1-ab77-6d18e8f9b2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:49.802430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa323d6-4093-4da1-ab77-6d18e8f9b2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:49.802810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa323d6-4093-4da1-ab77-6d18e8f9b2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:49.803179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa323d6-4093-4da1-ab77-6d18e8f9b2b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:49.805057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa323d6-4093-4da1-ab77-6d18e8f9b2b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:49.807432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa323d6-4093-4da1-ab77-6d18e8f9b2b2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:49.833250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1095/2174] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:49 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:49.837534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd859-bc39-435b-ac46-ba9bb93251ff admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:49.856201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd859-bc39-435b-ac46-ba9bb93251ff admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:49.862552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b778bfa-b968-4cfc-a001-9539eefae5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:49.863356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b778bfa-b968-4cfc-a001-9539eefae5df admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:49.863630 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b778bfa-b968-4cfc-a001-9539eefae5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:49.863899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b778bfa-b968-4cfc-a001-9539eefae5df admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:49.864975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b778bfa-b968-4cfc-a001-9539eefae5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:49.866731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b778bfa-b968-4cfc-a001-9539eefae5df admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:49.878960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1080/2175] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:49.883290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa323d6-4093-4da1-ab77-6d18e8f9b2b2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:49.894908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa323d6-4093-4da1-ab77-6d18e8f9b2b2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:49.899862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682ec68-be24-4023-96fa-a2ddceada3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:49.900982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682ec68-be24-4023-96fa-a2ddceada3a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:49.901268 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682ec68-be24-4023-96fa-a2ddceada3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:49.901646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682ec68-be24-4023-96fa-a2ddceada3a2 admin admin] PATH_INFO: `/v3/projects/17cf3832459944abacf72169a8abd09a/users/8324178d3bbc4ddeb5d5873d2893a84b/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:49.918960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682ec68-be24-4023-96fa-a2ddceada3a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17cf3832459944abacf72169a8abd09a, user_id=8324178d3bbc4ddeb5d5873d2893a84b, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:49.921741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682ec68-be24-4023-96fa-a2ddceada3a2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:49.941782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1096/2176] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:49 2022] PUT /identity/v3/projects/17cf3832459944abacf72169a8abd09a/users/8324178d3bbc4ddeb5d5873d2893a84b/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:49.942664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b778bfa-b968-4cfc-a001-9539eefae5df admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:49.954295 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b778bfa-b968-4cfc-a001-9539eefae5df admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:50.028806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbb686-2392-4c2a-be4a-db6a83a22173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:50.029874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbb686-2392-4c2a-be4a-db6a83a22173 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:50.030599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbb686-2392-4c2a-be4a-db6a83a22173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:50.031740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbb686-2392-4c2a-be4a-db6a83a22173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:50.032769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fbb686-2392-4c2a-be4a-db6a83a22173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:50.034736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fbb686-2392-4c2a-be4a-db6a83a22173 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:50.055461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1081/2177] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:50.059545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682ec68-be24-4023-96fa-a2ddceada3a2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:50.071545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682ec68-be24-4023-96fa-a2ddceada3a2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:50.077166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681df767-f322-4a29-ae48-e3e8c773713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:50.078176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681df767-f322-4a29-ae48-e3e8c773713c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:50.078480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681df767-f322-4a29-ae48-e3e8c773713c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:50.078805 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681df767-f322-4a29-ae48-e3e8c773713c admin admin] PATH_INFO: `/v3/projects/17cf3832459944abacf72169a8abd09a/users/8324178d3bbc4ddeb5d5873d2893a84b/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:50.081350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681df767-f322-4a29-ae48-e3e8c773713c admin admin] RBAC: Authorizing `identity:create_grant(project_id=17cf3832459944abacf72169a8abd09a, user_id=8324178d3bbc4ddeb5d5873d2893a84b, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:50.084400 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681df767-f322-4a29-ae48-e3e8c773713c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:50.105385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1097/2178] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:50 2022] PUT /identity/v3/projects/17cf3832459944abacf72169a8abd09a/users/8324178d3bbc4ddeb5d5873d2893a84b/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:53.132465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbb686-2392-4c2a-be4a-db6a83a22173 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:53.135389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b5fbb686-2392-4c2a-be4a-db6a83a22173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:53.155607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbb686-2392-4c2a-be4a-db6a83a22173 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:53.236589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:53.240000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:53.240938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:53.241737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] PATH_INFO: `/v3/users/4570168001784e9eab4711bf23afb47b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:53.243911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4570168001784e9eab4711bf23afb47b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:53.246078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:53.313219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4570168001784e9eab4711bf23afb47b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:53.314298 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4570168001784e9eab4711bf23afb47b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:53.330474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4570168001784e9eab4711bf23afb47b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:53.340556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4570168001784e9eab4711bf23afb47b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:53.366117 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dc32ee62-734a-4ebd-8bd5-49aa7567dafb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4570168001784e9eab4711bf23afb47b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:53.375111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1082/2179] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:53 2022] DELETE /identity/v3/users/4570168001784e9eab4711bf23afb47b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:53.692822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681df767-f322-4a29-ae48-e3e8c773713c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:53.694619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-681df767-f322-4a29-ae48-e3e8c773713c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:53.730152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681df767-f322-4a29-ae48-e3e8c773713c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:53.826949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:53.827859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:53.828132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:53.828397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] PATH_INFO: `/v3/projects/3b376e970ee94348aa59b7e1792baa7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:53.829919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b376e970ee94348aa59b7e1792baa7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:53.832432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:53.913977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b376e970ee94348aa59b7e1792baa7e', 'request_id': 'req-94dbcf11-d663-41bf-83d4-45664e77556e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:53.923762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b376e970ee94348aa59b7e1792baa7e', 'request_id': 'req-94dbcf11-d663-41bf-83d4-45664e77556e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:53.931090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b376e970ee94348aa59b7e1792baa7e', 'request_id': 'req-94dbcf11-d663-41bf-83d4-45664e77556e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:53.952052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] The token cache is being invalidate because project 3b376e970ee94348aa59b7e1792baa7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:53.952640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:53.953809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94dbcf11-d663-41bf-83d4-45664e77556e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:53.956991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1098/2180] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:53 2022] DELETE /identity/v3/projects/3b376e970ee94348aa59b7e1792baa7e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:53.964970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ccfe8-fe89-4a37-a602-5104f2760b8c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:53.966369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ccfe8-fe89-4a37-a602-5104f2760b8c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:53.966964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ccfe8-fe89-4a37-a602-5104f2760b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:53.990054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-af9ccfe8-fe89-4a37-a602-5104f2760b8c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:53.992256 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-af9ccfe8-fe89-4a37-a602-5104f2760b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:54.101980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1083/2181] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:54.104545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5efa0-3c4f-42b2-8abe-5c17ff448f35 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:54.104989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5efa0-3c4f-42b2-8abe-5c17ff448f35 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:54.105597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5efa0-3c4f-42b2-8abe-5c17ff448f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:54.133546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-95f5efa0-3c4f-42b2-8abe-5c17ff448f35 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:54.134788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-95f5efa0-3c4f-42b2-8abe-5c17ff448f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:54.148525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ccfe8-fe89-4a37-a602-5104f2760b8c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:54.154829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1099/2182] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:54.161012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ccfe8-fe89-4a37-a602-5104f2760b8c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:54.167894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27357c-88f2-4141-98d0-5dead81e4119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:54.169165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27357c-88f2-4141-98d0-5dead81e4119 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:54.170269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27357c-88f2-4141-98d0-5dead81e4119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:54.170798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27357c-88f2-4141-98d0-5dead81e4119 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:54.171822 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27357c-88f2-4141-98d0-5dead81e4119 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:45:54.172353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27357c-88f2-4141-98d0-5dead81e4119 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:54.174820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27357c-88f2-4141-98d0-5dead81e4119 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:54.194461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5efa0-3c4f-42b2-8abe-5c17ff448f35 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:54.212957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1084/2183] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:45:54.216833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5efa0-3c4f-42b2-8abe-5c17ff448f35 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:54.227344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded1bb7-ed90-4f57-8616-61477fb30d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:54.228122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded1bb7-ed90-4f57-8616-61477fb30d32 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:54.228473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded1bb7-ed90-4f57-8616-61477fb30d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:54.228962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded1bb7-ed90-4f57-8616-61477fb30d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:54.229739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded1bb7-ed90-4f57-8616-61477fb30d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:54.232143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded1bb7-ed90-4f57-8616-61477fb30d32 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:54.271269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1100/2184] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:54 2022] POST /identity/v3/projects => generated 369 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:45:54.275519 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27357c-88f2-4141-98d0-5dead81e4119 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:54.289497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27357c-88f2-4141-98d0-5dead81e4119 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:54.363128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8b057-affe-4fd3-897e-37f27ec3305c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:54.364396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8b057-affe-4fd3-897e-37f27ec3305c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:54.364866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8b057-affe-4fd3-897e-37f27ec3305c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:54.365316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8b057-affe-4fd3-897e-37f27ec3305c admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:54.366168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8b057-affe-4fd3-897e-37f27ec3305c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:54.369589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8b057-affe-4fd3-897e-37f27ec3305c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:54.428563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1085/2185] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:54 2022] POST /identity/v3/users => generated 285 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:45:54.431230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded1bb7-ed90-4f57-8616-61477fb30d32 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:54.447608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded1bb7-ed90-4f57-8616-61477fb30d32 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:54.458469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbbb7f-cff3-4b1d-ac76-606d9ccd53b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:54.459169 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbbb7f-cff3-4b1d-ac76-606d9ccd53b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:54.459445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbbb7f-cff3-4b1d-ac76-606d9ccd53b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:54.459717 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbbb7f-cff3-4b1d-ac76-606d9ccd53b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:54.462884 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbbb7f-cff3-4b1d-ac76-606d9ccd53b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:54.469955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbbb7f-cff3-4b1d-ac76-606d9ccd53b0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:54.509800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1101/2186] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:54.513007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8b057-affe-4fd3-897e-37f27ec3305c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:54.532731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8b057-affe-4fd3-897e-37f27ec3305c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:54.543497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8dda3-a18a-4df2-8682-3c13e926e060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:54.545645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8dda3-a18a-4df2-8682-3c13e926e060 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:54.546079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8dda3-a18a-4df2-8682-3c13e926e060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:54.546356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8dda3-a18a-4df2-8682-3c13e926e060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:54.547125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8dda3-a18a-4df2-8682-3c13e926e060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:54.553175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8dda3-a18a-4df2-8682-3c13e926e060 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:54.632478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1086/2187] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:54.636773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbbb7f-cff3-4b1d-ac76-606d9ccd53b0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:54.662817 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbbb7f-cff3-4b1d-ac76-606d9ccd53b0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:54.695416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415fd38-18e0-4e27-8e1b-b2234a05d61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:54.698021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415fd38-18e0-4e27-8e1b-b2234a05d61c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:54.699021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415fd38-18e0-4e27-8e1b-b2234a05d61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:54.703073 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415fd38-18e0-4e27-8e1b-b2234a05d61c admin admin] PATH_INFO: `/v3/projects/c9e23b5101ab4c63841dded796de6bc9/users/f91f74416bf84fc5b0423d9ced235c7c/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:54.746235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415fd38-18e0-4e27-8e1b-b2234a05d61c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e23b5101ab4c63841dded796de6bc9, user_id=f91f74416bf84fc5b0423d9ced235c7c, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:54.749519 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415fd38-18e0-4e27-8e1b-b2234a05d61c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:54.787827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1102/2188] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:54 2022] PUT /identity/v3/projects/c9e23b5101ab4c63841dded796de6bc9/users/f91f74416bf84fc5b0423d9ced235c7c/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:54.800039 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d161c-f2d9-4c7e-85fb-f3e4c9bac27d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:54.800346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d161c-f2d9-4c7e-85fb-f3e4c9bac27d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:54.800669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d161c-f2d9-4c7e-85fb-f3e4c9bac27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:54.846427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ab6d161c-f2d9-4c7e-85fb-f3e4c9bac27d None None] MFA Rules not processed for user `f91f74416bf84fc5b0423d9ced235c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:54.847939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ab6d161c-f2d9-4c7e-85fb-f3e4c9bac27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:54.978802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1087/2189] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:54.983307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415fd38-18e0-4e27-8e1b-b2234a05d61c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:55.117790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415fd38-18e0-4e27-8e1b-b2234a05d61c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:55.127585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea70f7a-068a-4fea-bc9e-5dea0f646b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:55.129220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea70f7a-068a-4fea-bc9e-5dea0f646b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:55.129712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea70f7a-068a-4fea-bc9e-5dea0f646b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:55.130675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea70f7a-068a-4fea-bc9e-5dea0f646b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:55.131818 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea70f7a-068a-4fea-bc9e-5dea0f646b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:55.134658 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea70f7a-068a-4fea-bc9e-5dea0f646b23 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:55.184473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1103/2190] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:54 2022] POST /identity/v3/projects => generated 369 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:55.186384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d161c-f2d9-4c7e-85fb-f3e4c9bac27d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:55.204836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d161c-f2d9-4c7e-85fb-f3e4c9bac27d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:55.230870 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea70f7a-068a-4fea-bc9e-5dea0f646b23 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:55.233165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eea70f7a-068a-4fea-bc9e-5dea0f646b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:55.369649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea70f7a-068a-4fea-bc9e-5dea0f646b23 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:55.375907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58d1b4-f408-4a17-bb63-905ddb6f2a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:55.376675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58d1b4-f408-4a17-bb63-905ddb6f2a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:55.376961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58d1b4-f408-4a17-bb63-905ddb6f2a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:55.377313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58d1b4-f408-4a17-bb63-905ddb6f2a0d admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:55.377938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58d1b4-f408-4a17-bb63-905ddb6f2a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:55.380047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:55.381014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:55.381293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:55.381567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] PATH_INFO: `/v3/users/967e91053d4344fd90e37dea693fe75e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:55.383430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58d1b4-f408-4a17-bb63-905ddb6f2a0d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:55.383543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=967e91053d4344fd90e37dea693fe75e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:55.387808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:55.423497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1088/2191] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:55 2022] POST /identity/v3/users => generated 291 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:55.427602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58d1b4-f408-4a17-bb63-905ddb6f2a0d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:55.445711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58d1b4-f408-4a17-bb63-905ddb6f2a0d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:55.461740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51ccef-1704-4f16-9860-fbd8d8d2be76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:55.463701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51ccef-1704-4f16-9860-fbd8d8d2be76 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:55.463983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51ccef-1704-4f16-9860-fbd8d8d2be76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:55.464251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51ccef-1704-4f16-9860-fbd8d8d2be76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:55.469117 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51ccef-1704-4f16-9860-fbd8d8d2be76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:55.473372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51ccef-1704-4f16-9860-fbd8d8d2be76 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:55.496964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '967e91053d4344fd90e37dea693fe75e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:55.503159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1089/2192] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:45:55.504825 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '967e91053d4344fd90e37dea693fe75e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:55.506761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51ccef-1704-4f16-9860-fbd8d8d2be76 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:55.508649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1b51ccef-1704-4f16-9860-fbd8d8d2be76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:55.522445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '967e91053d4344fd90e37dea693fe75e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:55.538044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51ccef-1704-4f16-9860-fbd8d8d2be76 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:55.548591 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89442c09-e93f-4e7d-bba5-7dd4bbd0f476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:55.549572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442c09-e93f-4e7d-bba5-7dd4bbd0f476 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:55.549922 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442c09-e93f-4e7d-bba5-7dd4bbd0f476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:55.550289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442c09-e93f-4e7d-bba5-7dd4bbd0f476 admin admin] PATH_INFO: `/v3/projects/583efe9def77497d95c203eeb7b43ff1/users/3b7e4ec17c9a455596b68d77aeb8da9a/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:55.555901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '967e91053d4344fd90e37dea693fe75e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:55.570760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e1cb1880-7515-41c0-8d68-60baa7a6db50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '967e91053d4344fd90e37dea693fe75e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:55.583850 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1104/2193] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:55 2022] DELETE /identity/v3/users/967e91053d4344fd90e37dea693fe75e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:55.585490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a8cd4-854a-434f-af4f-8ff675604fee None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:55.585776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a8cd4-854a-434f-af4f-8ff675604fee None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:55.586054 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a8cd4-854a-434f-af4f-8ff675604fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:55.586394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89442c09-e93f-4e7d-bba5-7dd4bbd0f476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=583efe9def77497d95c203eeb7b43ff1, user_id=3b7e4ec17c9a455596b68d77aeb8da9a, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:55.589672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89442c09-e93f-4e7d-bba5-7dd4bbd0f476 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:55.622307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1090/2194] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:55 2022] PUT /identity/v3/projects/583efe9def77497d95c203eeb7b43ff1/users/3b7e4ec17c9a455596b68d77aeb8da9a/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:55.623552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89442c09-e93f-4e7d-bba5-7dd4bbd0f476 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:55.642506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-566a8cd4-854a-434f-af4f-8ff675604fee None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:55.644181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-566a8cd4-854a-434f-af4f-8ff675604fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:55.717899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89442c09-e93f-4e7d-bba5-7dd4bbd0f476 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:55.727512 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab61d81-c1f4-4437-b54d-f41a9242aeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:55.728300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab61d81-c1f4-4437-b54d-f41a9242aeab admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:55.729206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab61d81-c1f4-4437-b54d-f41a9242aeab admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:55.729206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab61d81-c1f4-4437-b54d-f41a9242aeab admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:55.730765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab61d81-c1f4-4437-b54d-f41a9242aeab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:55.735477 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab61d81-c1f4-4437-b54d-f41a9242aeab admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:55.738387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1105/2195] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:45:55.763187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1091/2196] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:55.770144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a8cd4-854a-434f-af4f-8ff675604fee None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:55.784236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab61d81-c1f4-4437-b54d-f41a9242aeab admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:55.790348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2ab61d81-c1f4-4437-b54d-f41a9242aeab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:55.793984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a8cd4-854a-434f-af4f-8ff675604fee None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:55.828618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771be800-3af7-4b1a-90f7-c1701c1af300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:55.832627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771be800-3af7-4b1a-90f7-c1701c1af300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:55.833512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771be800-3af7-4b1a-90f7-c1701c1af300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:55.833512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771be800-3af7-4b1a-90f7-c1701c1af300 admin admin] PATH_INFO: `/v3/projects/583efe9def77497d95c203eeb7b43ff1/users/3b7e4ec17c9a455596b68d77aeb8da9a/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:55.837327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771be800-3af7-4b1a-90f7-c1701c1af300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=583efe9def77497d95c203eeb7b43ff1, user_id=3b7e4ec17c9a455596b68d77aeb8da9a, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:55.839966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771be800-3af7-4b1a-90f7-c1701c1af300 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:55.874466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1106/2197] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:55 2022] PUT /identity/v3/projects/583efe9def77497d95c203eeb7b43ff1/users/3b7e4ec17c9a455596b68d77aeb8da9a/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:55.884978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a0c8-1a66-4c14-9164-cd5789c71b26 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:55.885344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a0c8-1a66-4c14-9164-cd5789c71b26 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:55.885689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a0c8-1a66-4c14-9164-cd5789c71b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:55.901431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab61d81-c1f4-4437-b54d-f41a9242aeab admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:55.924563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e2d8a0c8-1a66-4c14-9164-cd5789c71b26 None None] MFA Rules not processed for user `3b7e4ec17c9a455596b68d77aeb8da9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:55.927327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e2d8a0c8-1a66-4c14-9164-cd5789c71b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:55.982398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343a024-028b-43bd-be5b-17ed59ec32c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:55.983689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343a024-028b-43bd-be5b-17ed59ec32c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:55.983902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343a024-028b-43bd-be5b-17ed59ec32c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:55.984246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343a024-028b-43bd-be5b-17ed59ec32c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:56.041559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1107/2198] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:55 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:45:56.054437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8a0c8-1a66-4c14-9164-cd5789c71b26 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:56.056821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e2d8a0c8-1a66-4c14-9164-cd5789c71b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:56.069604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343a024-028b-43bd-be5b-17ed59ec32c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:56.071430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343a024-028b-43bd-be5b-17ed59ec32c1 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:56.087198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1092/2199] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:45:56.123687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8a0c8-1a66-4c14-9164-cd5789c71b26 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:56.127546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970fee6-c525-4b04-ba17-ac45adfdace1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:56.128209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970fee6-c525-4b04-ba17-ac45adfdace1 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:56.128510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970fee6-c525-4b04-ba17-ac45adfdace1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:56.128929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970fee6-c525-4b04-ba17-ac45adfdace1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:56.139989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970fee6-c525-4b04-ba17-ac45adfdace1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:56.141117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970fee6-c525-4b04-ba17-ac45adfdace1 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:56.151550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1108/2200] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:56 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:45:56.641299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343a024-028b-43bd-be5b-17ed59ec32c1 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:56.642896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0343a024-028b-43bd-be5b-17ed59ec32c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:56.666529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343a024-028b-43bd-be5b-17ed59ec32c1 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:56.686762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:56.687680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:56.687964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:56.688235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] PATH_INFO: `/v3/projects/e9ea298310964ff9ba31f06396998bc2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:56.694853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ea298310964ff9ba31f06396998bc2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:56.697059 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:56.785294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ea298310964ff9ba31f06396998bc2', 'request_id': 'req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:56.795950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ea298310964ff9ba31f06396998bc2', 'request_id': 'req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:56.804964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ea298310964ff9ba31f06396998bc2', 'request_id': 'req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:56.823313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] The token cache is being invalidate because project e9ea298310964ff9ba31f06396998bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:56.823970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:56.825324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-359a7b2f-e69e-47ce-9fdf-272257d8dd7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:56.827188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1093/2201] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:56 2022] DELETE /identity/v3/projects/e9ea298310964ff9ba31f06396998bc2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:56.832478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71c747-f8fd-4309-aec7-57edb9cfe878 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:56.833030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71c747-f8fd-4309-aec7-57edb9cfe878 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:56.833544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71c747-f8fd-4309-aec7-57edb9cfe878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:56.863122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9e71c747-f8fd-4309-aec7-57edb9cfe878 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:56.865483 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9e71c747-f8fd-4309-aec7-57edb9cfe878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:56.995372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1109/2202] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:45:57.003540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2ea67-b844-4bdd-a453-406d663436fe None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:57.003944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2ea67-b844-4bdd-a453-406d663436fe None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:57.004674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2ea67-b844-4bdd-a453-406d663436fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:57.028137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dbe2ea67-b844-4bdd-a453-406d663436fe None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:57.029839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dbe2ea67-b844-4bdd-a453-406d663436fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:57.047013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1094/2203] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:45:57.051547 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71c747-f8fd-4309-aec7-57edb9cfe878 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:57.072024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71c747-f8fd-4309-aec7-57edb9cfe878 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:57.084204 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268417ad-1682-4b4d-8b8e-188efa401ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:57.085040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268417ad-1682-4b4d-8b8e-188efa401ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:57.085455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268417ad-1682-4b4d-8b8e-188efa401ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:57.085732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268417ad-1682-4b4d-8b8e-188efa401ae9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:57.086488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268417ad-1682-4b4d-8b8e-188efa401ae9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:45:57.086770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268417ad-1682-4b4d-8b8e-188efa401ae9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:57.088476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268417ad-1682-4b4d-8b8e-188efa401ae9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:57.106221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1110/2204] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:45:57.110962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2ea67-b844-4bdd-a453-406d663436fe None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:57.121097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2ea67-b844-4bdd-a453-406d663436fe None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:57.127358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e7981-b236-4ce2-bb92-4a963befdc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:57.128221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e7981-b236-4ce2-bb92-4a963befdc2e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:57.129235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e7981-b236-4ce2-bb92-4a963befdc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:57.129672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e7981-b236-4ce2-bb92-4a963befdc2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:57.130551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e7981-b236-4ce2-bb92-4a963befdc2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:57.132680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e7981-b236-4ce2-bb92-4a963befdc2e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:57.191346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1095/2205] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:45:57 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:57.194678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268417ad-1682-4b4d-8b8e-188efa401ae9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:57.212621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268417ad-1682-4b4d-8b8e-188efa401ae9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:57.291710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa70ed3-37fc-435e-853b-9c706d05582f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:57.292325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa70ed3-37fc-435e-853b-9c706d05582f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:57.292641 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa70ed3-37fc-435e-853b-9c706d05582f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:57.292916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa70ed3-37fc-435e-853b-9c706d05582f admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:57.293616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa70ed3-37fc-435e-853b-9c706d05582f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:57.295830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa70ed3-37fc-435e-853b-9c706d05582f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:57.321511 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1111/2206] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:45:57 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:45:57.326342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e7981-b236-4ce2-bb92-4a963befdc2e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:57.340626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e7981-b236-4ce2-bb92-4a963befdc2e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:57.360707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e60e6-79b4-4169-b997-d53e6bd97cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:57.361478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e60e6-79b4-4169-b997-d53e6bd97cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:57.361923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e60e6-79b4-4169-b997-d53e6bd97cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:57.362165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e60e6-79b4-4169-b997-d53e6bd97cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:57.363551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e60e6-79b4-4169-b997-d53e6bd97cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:57.366913 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e60e6-79b4-4169-b997-d53e6bd97cf5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:57.385791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1096/2207] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:45:57.390133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa70ed3-37fc-435e-853b-9c706d05582f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:57.403114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa70ed3-37fc-435e-853b-9c706d05582f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:57.408702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc72ca8-1041-4adc-937e-8158255b754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:57.409634 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72ca8-1041-4adc-937e-8158255b754f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:57.409924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72ca8-1041-4adc-937e-8158255b754f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:57.410199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72ca8-1041-4adc-937e-8158255b754f admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:57.411373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc72ca8-1041-4adc-937e-8158255b754f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:57.413184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc72ca8-1041-4adc-937e-8158255b754f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:57.428347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1112/2208] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:45:57.432520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e60e6-79b4-4169-b997-d53e6bd97cf5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:57.447220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e60e6-79b4-4169-b997-d53e6bd97cf5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:57.453820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c4910-76fb-4b49-b593-f4f0aff9905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:57.454733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c4910-76fb-4b49-b593-f4f0aff9905b admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:57.455016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c4910-76fb-4b49-b593-f4f0aff9905b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:57.455440 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c4910-76fb-4b49-b593-f4f0aff9905b admin admin] PATH_INFO: `/v3/projects/f7bdcaacb5bd4224aecaa50a5fb6cc5f/users/0dcf8d9a9d84417ca2294670ada4f435/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:57.478456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c4910-76fb-4b49-b593-f4f0aff9905b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7bdcaacb5bd4224aecaa50a5fb6cc5f, user_id=0dcf8d9a9d84417ca2294670ada4f435, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:57.481357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c4910-76fb-4b49-b593-f4f0aff9905b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:57.502828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1097/2209] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:45:57 2022] PUT /identity/v3/projects/f7bdcaacb5bd4224aecaa50a5fb6cc5f/users/0dcf8d9a9d84417ca2294670ada4f435/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:57.515548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae532a5-8f9f-473f-98aa-d982cf1c8710 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:57.515548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae532a5-8f9f-473f-98aa-d982cf1c8710 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:57.515548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae532a5-8f9f-473f-98aa-d982cf1c8710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:57.593023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0ae532a5-8f9f-473f-98aa-d982cf1c8710 None None] MFA Rules not processed for user `0dcf8d9a9d84417ca2294670ada4f435`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:57.594301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0ae532a5-8f9f-473f-98aa-d982cf1c8710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:57.742348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1113/2210] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:45:57.813134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c4910-76fb-4b49-b593-f4f0aff9905b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:57.815061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e20c4910-76fb-4b49-b593-f4f0aff9905b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:57.879356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdea144-8f7d-477b-a396-a0c449eecc33 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:57.879862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdea144-8f7d-477b-a396-a0c449eecc33 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:57.879956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdea144-8f7d-477b-a396-a0c449eecc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:57.903058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c4910-76fb-4b49-b593-f4f0aff9905b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:57.909008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54b7bb-96ab-4001-8693-01e2b304d181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:57.910210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54b7bb-96ab-4001-8693-01e2b304d181 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:57.911128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54b7bb-96ab-4001-8693-01e2b304d181 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:57.911560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54b7bb-96ab-4001-8693-01e2b304d181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:57.917579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2cdea144-8f7d-477b-a396-a0c449eecc33 None None] MFA Rules not processed for user `8324178d3bbc4ddeb5d5873d2893a84b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:57.918804 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2cdea144-8f7d-477b-a396-a0c449eecc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:57.923748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54b7bb-96ab-4001-8693-01e2b304d181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:57.925759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54b7bb-96ab-4001-8693-01e2b304d181 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:57.936224 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1098/2211] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:58.000543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1114/2212] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:45:58.013893 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54b7bb-96ab-4001-8693-01e2b304d181 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:58.015317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5a54b7bb-96ab-4001-8693-01e2b304d181 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:58.108375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdea144-8f7d-477b-a396-a0c449eecc33 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:58.110465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2cdea144-8f7d-477b-a396-a0c449eecc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:58.142326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54b7bb-96ab-4001-8693-01e2b304d181 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:58.149138 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68868d62-6e4c-42de-a51c-cdf4b12d7b44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:58.149787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68868d62-6e4c-42de-a51c-cdf4b12d7b44 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:58.150246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68868d62-6e4c-42de-a51c-cdf4b12d7b44 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:58.150671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68868d62-6e4c-42de-a51c-cdf4b12d7b44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:58.152442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-68868d62-6e4c-42de-a51c-cdf4b12d7b44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:58.180254 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdea144-8f7d-477b-a396-a0c449eecc33 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:58.185090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:58.186041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:58.186326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:58.186597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] PATH_INFO: `/v3/users/f91f74416bf84fc5b0423d9ced235c7c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:58.187835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f91f74416bf84fc5b0423d9ced235c7c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:58.189801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:58.241137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68868d62-6e4c-42de-a51c-cdf4b12d7b44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:58.243102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68868d62-6e4c-42de-a51c-cdf4b12d7b44 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:58.254384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1099/2213] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:45:58.266830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f91f74416bf84fc5b0423d9ced235c7c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:58.267967 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f91f74416bf84fc5b0423d9ced235c7c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:58.284174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f91f74416bf84fc5b0423d9ced235c7c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:58.295425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f91f74416bf84fc5b0423d9ced235c7c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:58.303882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f91f74416bf84fc5b0423d9ced235c7c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:58.310966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1115/2214] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:58 2022] DELETE /identity/v3/users/f91f74416bf84fc5b0423d9ced235c7c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:45:58.315723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bf198-30bf-418a-a427-4475b6af6480 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:58.316077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bf198-30bf-418a-a427-4475b6af6480 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:58.316421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bf198-30bf-418a-a427-4475b6af6480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:58.343811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c96bf198-30bf-418a-a427-4475b6af6480 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:58.345240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c96bf198-30bf-418a-a427-4475b6af6480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:58.434550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1100/2215] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:45:58.444726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:58.446203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:58.520172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecf1d4-d0bf-4a79-8657-87c966507d7b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:58.529303 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2605baf-f280-4b36-829e-edbc62adf4f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:58.529987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605baf-f280-4b36-829e-edbc62adf4f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:58.530285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605baf-f280-4b36-829e-edbc62adf4f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:58.530698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605baf-f280-4b36-829e-edbc62adf4f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:58.549688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2605baf-f280-4b36-829e-edbc62adf4f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:58.551460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2605baf-f280-4b36-829e-edbc62adf4f4 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:58.562606 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1116/2216] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:45:59.008807 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bf198-30bf-418a-a427-4475b6af6480 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:59.010267 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c96bf198-30bf-418a-a427-4475b6af6480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:59.023760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bf198-30bf-418a-a427-4475b6af6480 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:59.028684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:59.030046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:59.030548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:59.031017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] PATH_INFO: `/v3/projects/c9e23b5101ab4c63841dded796de6bc9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:59.032526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e23b5101ab4c63841dded796de6bc9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:59.035232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:59.096221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e23b5101ab4c63841dded796de6bc9', 'request_id': 'req-b4fb21a7-5962-4a19-9147-17cc627dd511'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:59.104385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e23b5101ab4c63841dded796de6bc9', 'request_id': 'req-b4fb21a7-5962-4a19-9147-17cc627dd511'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:59.111566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e23b5101ab4c63841dded796de6bc9', 'request_id': 'req-b4fb21a7-5962-4a19-9147-17cc627dd511'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:59.122899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] The token cache is being invalidate because project c9e23b5101ab4c63841dded796de6bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:45:59.123211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:59.123880 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:59.125762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1101/2217] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:59 2022] DELETE /identity/v3/projects/c9e23b5101ab4c63841dded796de6bc9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:59.129498 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2605baf-f280-4b36-829e-edbc62adf4f4 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:59.139144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2605baf-f280-4b36-829e-edbc62adf4f4 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:59.225965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:59.226872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:59.227156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:59.227428 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] PATH_INFO: `/v3/users/3b7e4ec17c9a455596b68d77aeb8da9a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:59.228855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b7e4ec17c9a455596b68d77aeb8da9a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:59.230871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:59.292581 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b7e4ec17c9a455596b68d77aeb8da9a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:59.293586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b7e4ec17c9a455596b68d77aeb8da9a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:59.310751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b7e4ec17c9a455596b68d77aeb8da9a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:59.319793 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b7e4ec17c9a455596b68d77aeb8da9a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:59.327669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d2008822-540d-42e1-ae2b-56a5ac4ddd71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b7e4ec17c9a455596b68d77aeb8da9a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:45:59.337915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1117/2218] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:45:59 2022] DELETE /identity/v3/users/3b7e4ec17c9a455596b68d77aeb8da9a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:45:59.775520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:45:59.776797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:59.797608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb21a7-5962-4a19-9147-17cc627dd511 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:45:59.803072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8076fbb-c606-4b65-9ba8-d55891fa284d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:59.803355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8076fbb-c606-4b65-9ba8-d55891fa284d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:59.803628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8076fbb-c606-4b65-9ba8-d55891fa284d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:59.827143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d8076fbb-c606-4b65-9ba8-d55891fa284d None None] MFA Rules not processed for user `0ad000d2e49f4b6fa0e9315d35e7d4da`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:45:59.830037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d8076fbb-c606-4b65-9ba8-d55891fa284d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:45:59.918945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:45:59.920200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:45:59.920855 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:45:59.921334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] PATH_INFO: `/v3/projects/583efe9def77497d95c203eeb7b43ff1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:45:59.923219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=583efe9def77497d95c203eeb7b43ff1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:45:59.925574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:45:59.951806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1118/2219] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:45:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:45:59.997100 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583efe9def77497d95c203eeb7b43ff1', 'request_id': 'req-25ad0cbc-4075-4b06-b438-9126b7844ac2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:00.009025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583efe9def77497d95c203eeb7b43ff1', 'request_id': 'req-25ad0cbc-4075-4b06-b438-9126b7844ac2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:00.020794 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583efe9def77497d95c203eeb7b43ff1', 'request_id': 'req-25ad0cbc-4075-4b06-b438-9126b7844ac2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:00.040306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] The token cache is being invalidate because project 583efe9def77497d95c203eeb7b43ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:46:00.040741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:00.041508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25ad0cbc-4075-4b06-b438-9126b7844ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:00.044541 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1102/2220] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:45:59 2022] DELETE /identity/v3/projects/583efe9def77497d95c203eeb7b43ff1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:00.051495 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83ebe0-3c8d-4034-b042-d456e0869b49 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:00.053546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83ebe0-3c8d-4034-b042-d456e0869b49 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:00.053654 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83ebe0-3c8d-4034-b042-d456e0869b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:00.088201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9e83ebe0-3c8d-4034-b042-d456e0869b49 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:00.089759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9e83ebe0-3c8d-4034-b042-d456e0869b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:00.185610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1119/2221] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:00.195432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6bd17-574a-4ffd-98b0-37f59c3b7f42 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:00.196349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6bd17-574a-4ffd-98b0-37f59c3b7f42 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:00.196988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6bd17-574a-4ffd-98b0-37f59c3b7f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:00.220773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b6f6bd17-574a-4ffd-98b0-37f59c3b7f42 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:00.222396 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b6f6bd17-574a-4ffd-98b0-37f59c3b7f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:00.233332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1103/2222] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:00.238095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83ebe0-3c8d-4034-b042-d456e0869b49 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:00.250894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83ebe0-3c8d-4034-b042-d456e0869b49 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:00.255776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fa49b-40c8-473d-8eda-e96b877f2366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:00.256455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fa49b-40c8-473d-8eda-e96b877f2366 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:00.256775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fa49b-40c8-473d-8eda-e96b877f2366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:00.257042 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fa49b-40c8-473d-8eda-e96b877f2366 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:00.257782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fa49b-40c8-473d-8eda-e96b877f2366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:46:00.258077 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fa49b-40c8-473d-8eda-e96b877f2366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:00.260158 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fa49b-40c8-473d-8eda-e96b877f2366 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:00.274859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1120/2223] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:46:00.280199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6bd17-574a-4ffd-98b0-37f59c3b7f42 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:00.289727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6bd17-574a-4ffd-98b0-37f59c3b7f42 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:00.295482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57684b33-4600-4ebd-9d57-b927753b2b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:00.296348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684b33-4600-4ebd-9d57-b927753b2b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:00.296865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684b33-4600-4ebd-9d57-b927753b2b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:00.297144 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684b33-4600-4ebd-9d57-b927753b2b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:00.297953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57684b33-4600-4ebd-9d57-b927753b2b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:00.300313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57684b33-4600-4ebd-9d57-b927753b2b7b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:00.339578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1104/2224] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:46:00 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:00.341433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fa49b-40c8-473d-8eda-e96b877f2366 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:00.357882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fa49b-40c8-473d-8eda-e96b877f2366 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:00.463119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f277a-57a2-4fa0-9567-f6f3d23076ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:00.463119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f277a-57a2-4fa0-9567-f6f3d23076ac admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:00.463119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f277a-57a2-4fa0-9567-f6f3d23076ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:00.463119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f277a-57a2-4fa0-9567-f6f3d23076ac admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:00.463409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f277a-57a2-4fa0-9567-f6f3d23076ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:00.463409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f277a-57a2-4fa0-9567-f6f3d23076ac admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:00.489238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1121/2225] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:46:00 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:00.495325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57684b33-4600-4ebd-9d57-b927753b2b7b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:00.522382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57684b33-4600-4ebd-9d57-b927753b2b7b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:00.539772 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223112ec-6640-4855-9e12-c541996a5e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:00.540616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223112ec-6640-4855-9e12-c541996a5e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:00.542088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223112ec-6640-4855-9e12-c541996a5e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:00.546052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223112ec-6640-4855-9e12-c541996a5e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:00.549132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223112ec-6640-4855-9e12-c541996a5e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:00.554152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223112ec-6640-4855-9e12-c541996a5e16 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:00.575137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1105/2226] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:00.579222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f277a-57a2-4fa0-9567-f6f3d23076ac admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:00.589779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f277a-57a2-4fa0-9567-f6f3d23076ac admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:00.612677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc42dc-df25-4025-a121-3909fb92c08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:00.613456 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc42dc-df25-4025-a121-3909fb92c08f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:00.613799 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc42dc-df25-4025-a121-3909fb92c08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:00.614157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc42dc-df25-4025-a121-3909fb92c08f admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:00.614991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc42dc-df25-4025-a121-3909fb92c08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:00.617196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc42dc-df25-4025-a121-3909fb92c08f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:00.635828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1122/2227] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:00.637575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223112ec-6640-4855-9e12-c541996a5e16 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:00.649202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223112ec-6640-4855-9e12-c541996a5e16 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:00.654565 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998ea6f-ca45-4177-b8f7-71d5f49ba81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:00.655468 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998ea6f-ca45-4177-b8f7-71d5f49ba81a admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:00.655764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998ea6f-ca45-4177-b8f7-71d5f49ba81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:00.656026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998ea6f-ca45-4177-b8f7-71d5f49ba81a admin admin] PATH_INFO: `/v3/projects/c7756d58822c4c69961977cbb760ac9e/users/d583143387df488896ca37a6a6f25b15/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:00.678177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998ea6f-ca45-4177-b8f7-71d5f49ba81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7756d58822c4c69961977cbb760ac9e, user_id=d583143387df488896ca37a6a6f25b15, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:00.683969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998ea6f-ca45-4177-b8f7-71d5f49ba81a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:00.699592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1106/2228] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:00 2022] PUT /identity/v3/projects/c7756d58822c4c69961977cbb760ac9e/users/d583143387df488896ca37a6a6f25b15/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:46:00.711049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56983d6-aaed-438e-9d0d-3c5ee6f76b52 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:00.711542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56983d6-aaed-438e-9d0d-3c5ee6f76b52 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:00.712266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56983d6-aaed-438e-9d0d-3c5ee6f76b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:00.745225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f56983d6-aaed-438e-9d0d-3c5ee6f76b52 None None] MFA Rules not processed for user `d583143387df488896ca37a6a6f25b15`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:00.746667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f56983d6-aaed-438e-9d0d-3c5ee6f76b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:00.852596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1123/2229] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:00.857583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998ea6f-ca45-4177-b8f7-71d5f49ba81a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:00.938903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998ea6f-ca45-4177-b8f7-71d5f49ba81a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:00.959953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3baa4-25fd-4d3a-88e1-0d013dfacd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:00.960811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3baa4-25fd-4d3a-88e1-0d013dfacd33 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:00.961043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3baa4-25fd-4d3a-88e1-0d013dfacd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:00.961319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3baa4-25fd-4d3a-88e1-0d013dfacd33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:00.962035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3baa4-25fd-4d3a-88e1-0d013dfacd33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:00.985870 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3baa4-25fd-4d3a-88e1-0d013dfacd33 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:01.050932 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1107/2230] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:46:00 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:46:01.055572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56983d6-aaed-438e-9d0d-3c5ee6f76b52 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:01.078924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56983d6-aaed-438e-9d0d-3c5ee6f76b52 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:01.167739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc450a-dc76-4c65-95d6-407662150d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:01.168919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc450a-dc76-4c65-95d6-407662150d7a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:01.169719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc450a-dc76-4c65-95d6-407662150d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:01.171498 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc450a-dc76-4c65-95d6-407662150d7a admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:01.172419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc450a-dc76-4c65-95d6-407662150d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:01.174808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc450a-dc76-4c65-95d6-407662150d7a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:01.224769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1124/2231] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:46:01 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:01.229838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3baa4-25fd-4d3a-88e1-0d013dfacd33 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:01.254429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3baa4-25fd-4d3a-88e1-0d013dfacd33 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:01.265366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699384ba-3217-4008-a6bf-f2dee2497002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:01.266062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699384ba-3217-4008-a6bf-f2dee2497002 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:01.266337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699384ba-3217-4008-a6bf-f2dee2497002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:01.266608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699384ba-3217-4008-a6bf-f2dee2497002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:01.267334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699384ba-3217-4008-a6bf-f2dee2497002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:01.269151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699384ba-3217-4008-a6bf-f2dee2497002 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:01.282509 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1108/2232] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:01.286944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc450a-dc76-4c65-95d6-407662150d7a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:01.298259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc450a-dc76-4c65-95d6-407662150d7a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:01.305572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47211317-b937-42e8-99b1-38fce8c6f5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:01.306269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47211317-b937-42e8-99b1-38fce8c6f5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:01.306652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47211317-b937-42e8-99b1-38fce8c6f5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:01.306918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47211317-b937-42e8-99b1-38fce8c6f5e3 admin admin] PATH_INFO: `/v3/projects/fe8a00a03a3d4e6c8ffec7ca7772208f/users/041ccbe062824975affc562060ed1515/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:01.323298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47211317-b937-42e8-99b1-38fce8c6f5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8a00a03a3d4e6c8ffec7ca7772208f, user_id=041ccbe062824975affc562060ed1515, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:01.326931 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47211317-b937-42e8-99b1-38fce8c6f5e3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:01.344540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1125/2233] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:01 2022] PUT /identity/v3/projects/fe8a00a03a3d4e6c8ffec7ca7772208f/users/041ccbe062824975affc562060ed1515/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:46:01.347374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699384ba-3217-4008-a6bf-f2dee2497002 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:01.358227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699384ba-3217-4008-a6bf-f2dee2497002 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:01.429842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f30c6-0af6-4118-b9ea-21c36c4d6330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:01.430549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f30c6-0af6-4118-b9ea-21c36c4d6330 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:01.430834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f30c6-0af6-4118-b9ea-21c36c4d6330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:01.431112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f30c6-0af6-4118-b9ea-21c36c4d6330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:01.431834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f30c6-0af6-4118-b9ea-21c36c4d6330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:01.433659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f30c6-0af6-4118-b9ea-21c36c4d6330 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:01.447158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1109/2234] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:01.451445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47211317-b937-42e8-99b1-38fce8c6f5e3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:01.462421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47211317-b937-42e8-99b1-38fce8c6f5e3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:01.468954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c01c3e-3ab6-440c-9b74-829f9b44344a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:01.470149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c01c3e-3ab6-440c-9b74-829f9b44344a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:01.470500 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c01c3e-3ab6-440c-9b74-829f9b44344a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:01.470844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c01c3e-3ab6-440c-9b74-829f9b44344a admin admin] PATH_INFO: `/v3/projects/fe8a00a03a3d4e6c8ffec7ca7772208f/users/041ccbe062824975affc562060ed1515/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:01.472986 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c01c3e-3ab6-440c-9b74-829f9b44344a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8a00a03a3d4e6c8ffec7ca7772208f, user_id=041ccbe062824975affc562060ed1515, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:01.476094 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c01c3e-3ab6-440c-9b74-829f9b44344a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:01.489982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1126/2235] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:01 2022] PUT /identity/v3/projects/fe8a00a03a3d4e6c8ffec7ca7772208f/users/041ccbe062824975affc562060ed1515/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:46:01.501479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a798-439c-4e29-8d3c-0d14cf1bce93 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:01.501793 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a798-439c-4e29-8d3c-0d14cf1bce93 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:01.502129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a798-439c-4e29-8d3c-0d14cf1bce93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:01.533959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1a91a798-439c-4e29-8d3c-0d14cf1bce93 None None] MFA Rules not processed for user `041ccbe062824975affc562060ed1515`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:01.535193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1a91a798-439c-4e29-8d3c-0d14cf1bce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:01.609310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1110/2236] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:46:01.628205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c01c3e-3ab6-440c-9b74-829f9b44344a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:01.629251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a798-439c-4e29-8d3c-0d14cf1bce93 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:01.630180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-54c01c3e-3ab6-440c-9b74-829f9b44344a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:01.630735 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1a91a798-439c-4e29-8d3c-0d14cf1bce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:01.712338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c01c3e-3ab6-440c-9b74-829f9b44344a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:01.717632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:01.718678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:01.719025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:01.719367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] PATH_INFO: `/v3/users/0dcf8d9a9d84417ca2294670ada4f435` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:01.720862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dcf8d9a9d84417ca2294670ada4f435)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:01.723089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:01.730852 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a798-439c-4e29-8d3c-0d14cf1bce93 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:01.736444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8580933-aa4f-4d9d-b92b-a8b9cd1365e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:01.737316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8580933-aa4f-4d9d-b92b-a8b9cd1365e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:01.737704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8580933-aa4f-4d9d-b92b-a8b9cd1365e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:01.738248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8580933-aa4f-4d9d-b92b-a8b9cd1365e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:01.748478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8580933-aa4f-4d9d-b92b-a8b9cd1365e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:01.749926 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8580933-aa4f-4d9d-b92b-a8b9cd1365e3 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:01.761801 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1111/2237] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:46:01.780289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dcf8d9a9d84417ca2294670ada4f435'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:01.781084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dcf8d9a9d84417ca2294670ada4f435'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:01.799847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dcf8d9a9d84417ca2294670ada4f435'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:01.811508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dcf8d9a9d84417ca2294670ada4f435'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:01.822085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dcf8d9a9d84417ca2294670ada4f435'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:01.829780 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1127/2238] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:46:01 2022] DELETE /identity/v3/users/0dcf8d9a9d84417ca2294670ada4f435 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:01.835438 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616ae06-1c69-42f5-8f8f-c84c196e21ac None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:01.835723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616ae06-1c69-42f5-8f8f-c84c196e21ac None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:01.835992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616ae06-1c69-42f5-8f8f-c84c196e21ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:01.845147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:01.847232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:01.857417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5616ae06-1c69-42f5-8f8f-c84c196e21ac None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:01.858782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5616ae06-1c69-42f5-8f8f-c84c196e21ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:01.908360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba1eff-8195-4b25-a30b-ca99b47206a2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:01.915309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcb57f-dd26-46e8-a473-e43a827318ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:01.916307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcb57f-dd26-46e8-a473-e43a827318ee service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:01.916860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcb57f-dd26-46e8-a473-e43a827318ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:01.917147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcb57f-dd26-46e8-a473-e43a827318ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:01.919068 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-46bcb57f-dd26-46e8-a473-e43a827318ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:01.923686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1112/2239] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:01.933996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616ae06-1c69-42f5-8f8f-c84c196e21ac None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:01.937458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5616ae06-1c69-42f5-8f8f-c84c196e21ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:01.992511 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcb57f-dd26-46e8-a473-e43a827318ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:01.993991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcb57f-dd26-46e8-a473-e43a827318ee service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:02.002200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616ae06-1c69-42f5-8f8f-c84c196e21ac None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:02.002957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1128/2240] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:46:02.006292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94940ffb-f39d-446f-880e-b91517f627f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:02.007105 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940ffb-f39d-446f-880e-b91517f627f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:02.007386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940ffb-f39d-446f-880e-b91517f627f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:02.007665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940ffb-f39d-446f-880e-b91517f627f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:02.017424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94940ffb-f39d-446f-880e-b91517f627f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:02.019168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94940ffb-f39d-446f-880e-b91517f627f7 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:02.029947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1113/2241] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:46:02.450317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcb57f-dd26-46e8-a473-e43a827318ee service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:02.452369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-46bcb57f-dd26-46e8-a473-e43a827318ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:02.460608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94940ffb-f39d-446f-880e-b91517f627f7 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:02.462109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-94940ffb-f39d-446f-880e-b91517f627f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:02.476940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcb57f-dd26-46e8-a473-e43a827318ee service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:02.482191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:02.483518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:02.484328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:02.484725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] PATH_INFO: `/v3/users/d583143387df488896ca37a6a6f25b15` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:02.485070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94940ffb-f39d-446f-880e-b91517f627f7 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:02.485975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d583143387df488896ca37a6a6f25b15)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:02.488180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:02.489939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:02.491588 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:02.492123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:02.501218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] PATH_INFO: `/v3/projects/f7bdcaacb5bd4224aecaa50a5fb6cc5f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:02.503439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7bdcaacb5bd4224aecaa50a5fb6cc5f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:02.505546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:02.580867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd583143387df488896ca37a6a6f25b15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:02.581649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd583143387df488896ca37a6a6f25b15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:02.588091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7bdcaacb5bd4224aecaa50a5fb6cc5f', 'request_id': 'req-83396613-3fc0-4396-8c12-b2643009124a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:02.597154 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd583143387df488896ca37a6a6f25b15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:02.599347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7bdcaacb5bd4224aecaa50a5fb6cc5f', 'request_id': 'req-83396613-3fc0-4396-8c12-b2643009124a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:02.609419 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7bdcaacb5bd4224aecaa50a5fb6cc5f', 'request_id': 'req-83396613-3fc0-4396-8c12-b2643009124a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:02.610238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd583143387df488896ca37a6a6f25b15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:02.622610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-483ec996-da1c-4b75-92a9-00af7c8e99f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd583143387df488896ca37a6a6f25b15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:02.624625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1129/2242] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:46:02 2022] DELETE /identity/v3/users/d583143387df488896ca37a6a6f25b15 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:46:02.627853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] The token cache is being invalidate because project f7bdcaacb5bd4224aecaa50a5fb6cc5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:46:02.627853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:02.627853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:02.628812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1114/2243] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:46:02 2022] DELETE /identity/v3/projects/f7bdcaacb5bd4224aecaa50a5fb6cc5f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:46:02.629759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba9ce2-7b40-4517-89da-ca9a3bf697bb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:02.630104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba9ce2-7b40-4517-89da-ca9a3bf697bb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:02.630437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba9ce2-7b40-4517-89da-ca9a3bf697bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:02.656167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d0ba9ce2-7b40-4517-89da-ca9a3bf697bb None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:02.657558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d0ba9ce2-7b40-4517-89da-ca9a3bf697bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:02.742743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1130/2244] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:02.751761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:02.753362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:02.831912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83396613-3fc0-4396-8c12-b2643009124a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:02.883306 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be1940-eb18-4af1-adb0-02d2a57688fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:02.884188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be1940-eb18-4af1-adb0-02d2a57688fe service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:02.884596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be1940-eb18-4af1-adb0-02d2a57688fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:02.884989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be1940-eb18-4af1-adb0-02d2a57688fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:02.896195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be1940-eb18-4af1-adb0-02d2a57688fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:02.897698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be1940-eb18-4af1-adb0-02d2a57688fe service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:02.907995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1115/2245] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:46:03.191482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba9ce2-7b40-4517-89da-ca9a3bf697bb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:03.193006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d0ba9ce2-7b40-4517-89da-ca9a3bf697bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:03.205491 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba9ce2-7b40-4517-89da-ca9a3bf697bb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:03.210560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:03.211500 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:03.211838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:03.212171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] PATH_INFO: `/v3/projects/c7756d58822c4c69961977cbb760ac9e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:03.213454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7756d58822c4c69961977cbb760ac9e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:03.215443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:03.261067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7756d58822c4c69961977cbb760ac9e', 'request_id': 'req-59efbb11-5fc8-4777-8809-605dbb254ead'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.270353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7756d58822c4c69961977cbb760ac9e', 'request_id': 'req-59efbb11-5fc8-4777-8809-605dbb254ead'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.276927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7756d58822c4c69961977cbb760ac9e', 'request_id': 'req-59efbb11-5fc8-4777-8809-605dbb254ead'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.287842 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] The token cache is being invalidate because project c7756d58822c4c69961977cbb760ac9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:46:03.288121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.288777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.290408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1131/2246] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:46:03 2022] DELETE /identity/v3/projects/c7756d58822c4c69961977cbb760ac9e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:03.294304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be1940-eb18-4af1-adb0-02d2a57688fe service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:03.304791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be1940-eb18-4af1-adb0-02d2a57688fe service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:03.364013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:03.364849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:03.365127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:03.365447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] PATH_INFO: `/v3/users/041ccbe062824975affc562060ed1515` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:03.366641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] RBAC: Authorizing `identity:delete_user(user_id=041ccbe062824975affc562060ed1515)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:03.368718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:03.418607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041ccbe062824975affc562060ed1515'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.419452 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041ccbe062824975affc562060ed1515'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.431722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041ccbe062824975affc562060ed1515'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.438173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041ccbe062824975affc562060ed1515'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.447576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c668307b-43df-4638-b9ae-c9c0515a273e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041ccbe062824975affc562060ed1515'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.454315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1116/2247] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:46:03 2022] DELETE /identity/v3/users/041ccbe062824975affc562060ed1515 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:03.743121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:03.744628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:03.760180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efbb11-5fc8-4777-8809-605dbb254ead admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:03.817186 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:03.818062 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:03.818345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:03.818618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] PATH_INFO: `/v3/projects/fe8a00a03a3d4e6c8ffec7ca7772208f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:03.819820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe8a00a03a3d4e6c8ffec7ca7772208f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:03.821766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:03.875454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe8a00a03a3d4e6c8ffec7ca7772208f', 'request_id': 'req-3fde3a50-bb1b-4378-be40-5b77e52a7807'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.882746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe8a00a03a3d4e6c8ffec7ca7772208f', 'request_id': 'req-3fde3a50-bb1b-4378-be40-5b77e52a7807'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.891271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe8a00a03a3d4e6c8ffec7ca7772208f', 'request_id': 'req-3fde3a50-bb1b-4378-be40-5b77e52a7807'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.902921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] The token cache is being invalidate because project fe8a00a03a3d4e6c8ffec7ca7772208f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:46:03.903201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.904673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3fde3a50-bb1b-4378-be40-5b77e52a7807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:03.906035 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1132/2248] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:46:03 2022] DELETE /identity/v3/projects/fe8a00a03a3d4e6c8ffec7ca7772208f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:03.911477 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0bf7a-63be-4640-8206-d61c7160c35f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:03.912106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0bf7a-63be-4640-8206-d61c7160c35f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:03.912106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0bf7a-63be-4640-8206-d61c7160c35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:03.932280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a9e0bf7a-63be-4640-8206-d61c7160c35f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:03.933552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a9e0bf7a-63be-4640-8206-d61c7160c35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:03.998362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1117/2249] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:04.007370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b12a5-2c1f-41d1-9f8c-b5e50f808d43 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.007653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b12a5-2c1f-41d1-9f8c-b5e50f808d43 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.007921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b12a5-2c1f-41d1-9f8c-b5e50f808d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.026577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ba5b12a5-2c1f-41d1-9f8c-b5e50f808d43 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:04.027782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ba5b12a5-2c1f-41d1-9f8c-b5e50f808d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:04.036584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1133/2250] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:04.040924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0bf7a-63be-4640-8206-d61c7160c35f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.053067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0bf7a-63be-4640-8206-d61c7160c35f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.058144 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a1bb2-ba02-4352-820a-e494b3233f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.058824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a1bb2-ba02-4352-820a-e494b3233f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.059104 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a1bb2-ba02-4352-820a-e494b3233f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.059377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a1bb2-ba02-4352-820a-e494b3233f45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.060132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a1bb2-ba02-4352-820a-e494b3233f45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:46:04.060522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a1bb2-ba02-4352-820a-e494b3233f45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.062848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a1bb2-ba02-4352-820a-e494b3233f45 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.079590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1118/2251] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:46:04.080798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b12a5-2c1f-41d1-9f8c-b5e50f808d43 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.090059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b12a5-2c1f-41d1-9f8c-b5e50f808d43 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.094939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f585a-ff9c-48d9-b053-a2b6993eca5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.095597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f585a-ff9c-48d9-b053-a2b6993eca5e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.095878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f585a-ff9c-48d9-b053-a2b6993eca5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.096152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f585a-ff9c-48d9-b053-a2b6993eca5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.096935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f585a-ff9c-48d9-b053-a2b6993eca5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.098842 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f585a-ff9c-48d9-b053-a2b6993eca5e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.125888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1134/2252] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:46:04 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:04.130130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a1bb2-ba02-4352-820a-e494b3233f45 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.140365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a1bb2-ba02-4352-820a-e494b3233f45 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.201413 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b4b59-f8a1-42f6-871c-85a9cdcd4ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.204084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b4b59-f8a1-42f6-871c-85a9cdcd4ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.204394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b4b59-f8a1-42f6-871c-85a9cdcd4ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.204697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b4b59-f8a1-42f6-871c-85a9cdcd4ba8 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.205447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b4b59-f8a1-42f6-871c-85a9cdcd4ba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.207395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b4b59-f8a1-42f6-871c-85a9cdcd4ba8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.237085 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1119/2253] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:46:04 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:04.238387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f585a-ff9c-48d9-b053-a2b6993eca5e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.249093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f585a-ff9c-48d9-b053-a2b6993eca5e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.254363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaa5e5-1a7f-472d-8657-697e3e87d349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.255224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaa5e5-1a7f-472d-8657-697e3e87d349 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.255594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaa5e5-1a7f-472d-8657-697e3e87d349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.255931 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaa5e5-1a7f-472d-8657-697e3e87d349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.256821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eaa5e5-1a7f-472d-8657-697e3e87d349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.258482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eaa5e5-1a7f-472d-8657-697e3e87d349 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.269233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1135/2254] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:04.273748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b4b59-f8a1-42f6-871c-85a9cdcd4ba8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.284409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b4b59-f8a1-42f6-871c-85a9cdcd4ba8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.288993 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ca0be-884c-4286-ab55-2f3a5a1bcc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.289730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ca0be-884c-4286-ab55-2f3a5a1bcc26 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.290019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ca0be-884c-4286-ab55-2f3a5a1bcc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.290285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ca0be-884c-4286-ab55-2f3a5a1bcc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.291000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ca0be-884c-4286-ab55-2f3a5a1bcc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.292963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ca0be-884c-4286-ab55-2f3a5a1bcc26 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.306665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1120/2255] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:04.307794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaa5e5-1a7f-472d-8657-697e3e87d349 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.317880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaa5e5-1a7f-472d-8657-697e3e87d349 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.323206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473e7be-e04b-4d4f-a794-4d70dbf61db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.324106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473e7be-e04b-4d4f-a794-4d70dbf61db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.324391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473e7be-e04b-4d4f-a794-4d70dbf61db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.325059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473e7be-e04b-4d4f-a794-4d70dbf61db9 admin admin] PATH_INFO: `/v3/projects/1bb5ce8bb1e24fb28a61390f5318e09b/users/604a7db8329945d7878ec012efa4ae7f/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.340232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473e7be-e04b-4d4f-a794-4d70dbf61db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb5ce8bb1e24fb28a61390f5318e09b, user_id=604a7db8329945d7878ec012efa4ae7f, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.342996 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473e7be-e04b-4d4f-a794-4d70dbf61db9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.356733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1136/2256] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:04 2022] PUT /identity/v3/projects/1bb5ce8bb1e24fb28a61390f5318e09b/users/604a7db8329945d7878ec012efa4ae7f/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:04.367626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4bef-066f-4f67-8753-286331402b62 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.368063 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4bef-066f-4f67-8753-286331402b62 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.368281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4bef-066f-4f67-8753-286331402b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.398605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-316b4bef-066f-4f67-8753-286331402b62 None None] MFA Rules not processed for user `604a7db8329945d7878ec012efa4ae7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:04.399807 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-316b4bef-066f-4f67-8753-286331402b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:04.463905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1121/2257] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:04.469013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473e7be-e04b-4d4f-a794-4d70dbf61db9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.538646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473e7be-e04b-4d4f-a794-4d70dbf61db9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.543809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e5c7b-f0e6-4f31-b1dc-68c4e5a50a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.544475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5c7b-f0e6-4f31-b1dc-68c4e5a50a9b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.544990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5c7b-f0e6-4f31-b1dc-68c4e5a50a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.545272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5c7b-f0e6-4f31-b1dc-68c4e5a50a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.545996 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e5c7b-f0e6-4f31-b1dc-68c4e5a50a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.548031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e5c7b-f0e6-4f31-b1dc-68c4e5a50a9b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.600691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b4bef-066f-4f67-8753-286331402b62 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.601074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1137/2258] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:46:04 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:04.619660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b4bef-066f-4f67-8753-286331402b62 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.691646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd66003-9e33-4b15-9cdd-c1d9f43b89dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.693650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd66003-9e33-4b15-9cdd-c1d9f43b89dc admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.694319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd66003-9e33-4b15-9cdd-c1d9f43b89dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.694596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd66003-9e33-4b15-9cdd-c1d9f43b89dc admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.695881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd66003-9e33-4b15-9cdd-c1d9f43b89dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.698379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd66003-9e33-4b15-9cdd-c1d9f43b89dc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.724920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1122/2259] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:46:04 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:04.726798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e5c7b-f0e6-4f31-b1dc-68c4e5a50a9b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.738631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e5c7b-f0e6-4f31-b1dc-68c4e5a50a9b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.744862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1abad-fa36-4e27-a1b5-02f01aba70be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.745664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1abad-fa36-4e27-a1b5-02f01aba70be admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.747759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1abad-fa36-4e27-a1b5-02f01aba70be admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.748310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1abad-fa36-4e27-a1b5-02f01aba70be admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.749247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1abad-fa36-4e27-a1b5-02f01aba70be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.751340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1abad-fa36-4e27-a1b5-02f01aba70be admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.768582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1138/2260] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:04.769890 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd66003-9e33-4b15-9cdd-c1d9f43b89dc admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.787282 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd66003-9e33-4b15-9cdd-c1d9f43b89dc admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.792910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c30a8-3bec-40c9-9ad1-e283ef8205a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.794985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c30a8-3bec-40c9-9ad1-e283ef8205a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.795351 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c30a8-3bec-40c9-9ad1-e283ef8205a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.795693 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c30a8-3bec-40c9-9ad1-e283ef8205a9 admin admin] PATH_INFO: `/v3/projects/32762bae60c64a69a1db011a496764a7/users/67e15bb0d79449ab9deaa3ba887831f1/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.817479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c30a8-3bec-40c9-9ad1-e283ef8205a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32762bae60c64a69a1db011a496764a7, user_id=67e15bb0d79449ab9deaa3ba887831f1, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.820366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c30a8-3bec-40c9-9ad1-e283ef8205a9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.841084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1123/2261] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:04 2022] PUT /identity/v3/projects/32762bae60c64a69a1db011a496764a7/users/67e15bb0d79449ab9deaa3ba887831f1/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:04.849151 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1abad-fa36-4e27-a1b5-02f01aba70be admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.864530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1abad-fa36-4e27-a1b5-02f01aba70be admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.926089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32db9aa-e924-4a6a-89cd-035844db8c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.926757 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32db9aa-e924-4a6a-89cd-035844db8c5c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.927035 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32db9aa-e924-4a6a-89cd-035844db8c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.927303 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32db9aa-e924-4a6a-89cd-035844db8c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.928048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32db9aa-e924-4a6a-89cd-035844db8c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.934307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32db9aa-e924-4a6a-89cd-035844db8c5c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.945349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1139/2262] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:04.949653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c30a8-3bec-40c9-9ad1-e283ef8205a9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:04.961147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c30a8-3bec-40c9-9ad1-e283ef8205a9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:04.966342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3235412-3b93-449c-9a13-7b32e9d07280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:04.967461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3235412-3b93-449c-9a13-7b32e9d07280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.967741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3235412-3b93-449c-9a13-7b32e9d07280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.968012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3235412-3b93-449c-9a13-7b32e9d07280 admin admin] PATH_INFO: `/v3/projects/32762bae60c64a69a1db011a496764a7/users/67e15bb0d79449ab9deaa3ba887831f1/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:04.970111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3235412-3b93-449c-9a13-7b32e9d07280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32762bae60c64a69a1db011a496764a7, user_id=67e15bb0d79449ab9deaa3ba887831f1, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:04.972765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3235412-3b93-449c-9a13-7b32e9d07280 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:04.983689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1124/2263] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:04 2022] PUT /identity/v3/projects/32762bae60c64a69a1db011a496764a7/users/67e15bb0d79449ab9deaa3ba887831f1/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:04.995721 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28479499-f75d-4141-9f82-bfd5cf40a9a1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:04.996010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28479499-f75d-4141-9f82-bfd5cf40a9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:04.996286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28479499-f75d-4141-9f82-bfd5cf40a9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:05.025386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-28479499-f75d-4141-9f82-bfd5cf40a9a1 None None] MFA Rules not processed for user `67e15bb0d79449ab9deaa3ba887831f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:05.026598 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-28479499-f75d-4141-9f82-bfd5cf40a9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:05.098994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1140/2264] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:05.112137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3235412-3b93-449c-9a13-7b32e9d07280 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:05.114283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a3235412-3b93-449c-9a13-7b32e9d07280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:05.174081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3235412-3b93-449c-9a13-7b32e9d07280 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:05.184534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef36c-8ab3-4ac1-9673-253ce96b7175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:05.185230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef36c-8ab3-4ac1-9673-253ce96b7175 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:05.186039 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef36c-8ab3-4ac1-9673-253ce96b7175 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:05.186348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef36c-8ab3-4ac1-9673-253ce96b7175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:05.252210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ef36c-8ab3-4ac1-9673-253ce96b7175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:05.254422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ef36c-8ab3-4ac1-9673-253ce96b7175 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:05.267258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1125/2265] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:46:09.919119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28479499-f75d-4141-9f82-bfd5cf40a9a1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:10.010288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28479499-f75d-4141-9f82-bfd5cf40a9a1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:10.016789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:10.018015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:10.018353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:10.018803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] PATH_INFO: `/v3/users/604a7db8329945d7878ec012efa4ae7f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:10.020812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] RBAC: Authorizing `identity:delete_user(user_id=604a7db8329945d7878ec012efa4ae7f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:10.023146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:10.079636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '604a7db8329945d7878ec012efa4ae7f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.080396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '604a7db8329945d7878ec012efa4ae7f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.093712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '604a7db8329945d7878ec012efa4ae7f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.102333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '604a7db8329945d7878ec012efa4ae7f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.110406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '604a7db8329945d7878ec012efa4ae7f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.118068 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1141/2266] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:46:09 2022] DELETE /identity/v3/users/604a7db8329945d7878ec012efa4ae7f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:10.122535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb12c2c-effa-4d5c-8fe8-f2eb3bd04b88 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:10.123034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb12c2c-effa-4d5c-8fe8-f2eb3bd04b88 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:10.123159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb12c2c-effa-4d5c-8fe8-f2eb3bd04b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:10.141219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0bb12c2c-effa-4d5c-8fe8-f2eb3bd04b88 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:10.142406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0bb12c2c-effa-4d5c-8fe8-f2eb3bd04b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:10.204854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1126/2267] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:46:10.214417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:10.217080 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:10.280722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f482c-a106-449b-afb7-6dec369dfaff admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:10.284984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ac44a-3768-4bd9-8784-26249e2b223d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:10.285726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ac44a-3768-4bd9-8784-26249e2b223d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:10.286077 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ac44a-3768-4bd9-8784-26249e2b223d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:10.286421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ac44a-3768-4bd9-8784-26249e2b223d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:10.297518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ac44a-3768-4bd9-8784-26249e2b223d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:10.298982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ac44a-3768-4bd9-8784-26249e2b223d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:10.312292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1142/2268] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:46:10.635947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb12c2c-effa-4d5c-8fe8-f2eb3bd04b88 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:10.640044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0bb12c2c-effa-4d5c-8fe8-f2eb3bd04b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:10.657504 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb12c2c-effa-4d5c-8fe8-f2eb3bd04b88 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:10.663159 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:10.664052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:10.664430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:10.664874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] PATH_INFO: `/v3/projects/1bb5ce8bb1e24fb28a61390f5318e09b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:10.666583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb5ce8bb1e24fb28a61390f5318e09b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:10.668694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:10.721484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb5ce8bb1e24fb28a61390f5318e09b', 'request_id': 'req-4b4e4efd-a629-4086-9266-861538b2d63a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.732087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb5ce8bb1e24fb28a61390f5318e09b', 'request_id': 'req-4b4e4efd-a629-4086-9266-861538b2d63a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.739452 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb5ce8bb1e24fb28a61390f5318e09b', 'request_id': 'req-4b4e4efd-a629-4086-9266-861538b2d63a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.752050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] The token cache is being invalidate because project 1bb5ce8bb1e24fb28a61390f5318e09b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:46:10.752334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.753086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.754738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1127/2269] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:46:10 2022] DELETE /identity/v3/projects/1bb5ce8bb1e24fb28a61390f5318e09b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:10.758826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ac44a-3768-4bd9-8784-26249e2b223d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:10.768998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ac44a-3768-4bd9-8784-26249e2b223d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:10.831497 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:10.832310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:10.832618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:10.832900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] PATH_INFO: `/v3/users/67e15bb0d79449ab9deaa3ba887831f1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:10.834287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] RBAC: Authorizing `identity:delete_user(user_id=67e15bb0d79449ab9deaa3ba887831f1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:10.836455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:10.885706 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67e15bb0d79449ab9deaa3ba887831f1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.886478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67e15bb0d79449ab9deaa3ba887831f1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.899539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67e15bb0d79449ab9deaa3ba887831f1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.917418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67e15bb0d79449ab9deaa3ba887831f1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.918854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1b0ec5d-cca7-481a-873e-b7dc7879801a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67e15bb0d79449ab9deaa3ba887831f1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:10.927403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1143/2270] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:46:10 2022] DELETE /identity/v3/users/67e15bb0d79449ab9deaa3ba887831f1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:11.212447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:11.213951 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:11.227372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e4efd-a629-4086-9266-861538b2d63a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:11.288538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:11.288538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.289432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.289432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] PATH_INFO: `/v3/projects/32762bae60c64a69a1db011a496764a7` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.290366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32762bae60c64a69a1db011a496764a7)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:11.293611 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:11.342654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32762bae60c64a69a1db011a496764a7', 'request_id': 'req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:11.350731 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32762bae60c64a69a1db011a496764a7', 'request_id': 'req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:11.357448 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32762bae60c64a69a1db011a496764a7', 'request_id': 'req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:11.367702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] The token cache is being invalidate because project 32762bae60c64a69a1db011a496764a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:46:11.368127 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:11.369035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e49ad295-3ad1-42bc-8be4-a23ffe2dffd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:11.370697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1128/2271] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:46:11 2022] DELETE /identity/v3/projects/32762bae60c64a69a1db011a496764a7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:11.375739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed621fe-27f0-4ffc-b7af-57b8c96099aa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.376023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed621fe-27f0-4ffc-b7af-57b8c96099aa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.376330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed621fe-27f0-4ffc-b7af-57b8c96099aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.394257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4ed621fe-27f0-4ffc-b7af-57b8c96099aa None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:11.395436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4ed621fe-27f0-4ffc-b7af-57b8c96099aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:11.457333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1144/2272] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:11.466030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c74bb-b8aa-44ca-908d-c18b91b2f9ac None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.466315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c74bb-b8aa-44ca-908d-c18b91b2f9ac None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.466589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c74bb-b8aa-44ca-908d-c18b91b2f9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.485443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-178c74bb-b8aa-44ca-908d-c18b91b2f9ac None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:11.486632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-178c74bb-b8aa-44ca-908d-c18b91b2f9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:11.495984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1129/2273] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:11.499401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed621fe-27f0-4ffc-b7af-57b8c96099aa None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:11.500823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c74bb-b8aa-44ca-908d-c18b91b2f9ac None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:11.502179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-178c74bb-b8aa-44ca-908d-c18b91b2f9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:11.509230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed621fe-27f0-4ffc-b7af-57b8c96099aa None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:11.513739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c74bb-b8aa-44ca-908d-c18b91b2f9ac None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:11.513936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1d5cf-6b51-45c4-8228-582e613b995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:11.514584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d5cf-6b51-45c4-8228-582e613b995c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.514869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d5cf-6b51-45c4-8228-582e613b995c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.515144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d5cf-6b51-45c4-8228-582e613b995c admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.515881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1d5cf-6b51-45c4-8228-582e613b995c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:46:11.516171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1d5cf-6b51-45c4-8228-582e613b995c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:11.517818 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1d5cf-6b51-45c4-8228-582e613b995c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:11.518618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:11.519474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.519747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.520010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] PATH_INFO: `/v3/users/fa4c1acb58fb4f5ba86b1668eb4dcd03` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.521208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa4c1acb58fb4f5ba86b1668eb4dcd03)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:11.523408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:11.529602 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1145/2274] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:46:11.539389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1d5cf-6b51-45c4-8228-582e613b995c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:11.550364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1d5cf-6b51-45c4-8228-582e613b995c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:11.557311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ee8b5-4a71-441b-abcf-e1bf4ad7cc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:11.558081 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ee8b5-4a71-441b-abcf-e1bf4ad7cc15 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.558325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ee8b5-4a71-441b-abcf-e1bf4ad7cc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.558720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ee8b5-4a71-441b-abcf-e1bf4ad7cc15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.559464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ee8b5-4a71-441b-abcf-e1bf4ad7cc15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:11.561431 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ee8b5-4a71-441b-abcf-e1bf4ad7cc15 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:11.574114 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa4c1acb58fb4f5ba86b1668eb4dcd03'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:11.575192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa4c1acb58fb4f5ba86b1668eb4dcd03'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:11.587682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa4c1acb58fb4f5ba86b1668eb4dcd03'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:11.590802 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1146/2275] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:46:11 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:46:11.594789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ee8b5-4a71-441b-abcf-e1bf4ad7cc15 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:11.596324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b83ee8b5-4a71-441b-abcf-e1bf4ad7cc15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:11.596434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa4c1acb58fb4f5ba86b1668eb4dcd03'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:11.605842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9f2ec925-b604-467b-9b08-d777b4217b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa4c1acb58fb4f5ba86b1668eb4dcd03'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:11.612520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1130/2276] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:46:11 2022] DELETE /identity/v3/users/fa4c1acb58fb4f5ba86b1668eb4dcd03 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:46:11.617052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed10da-accb-4b57-838a-20f1f99028fb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.617338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed10da-accb-4b57-838a-20f1f99028fb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.617643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed10da-accb-4b57-838a-20f1f99028fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.634807 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d2ed10da-accb-4b57-838a-20f1f99028fb None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:11.636005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d2ed10da-accb-4b57-838a-20f1f99028fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:11.666192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ee8b5-4a71-441b-abcf-e1bf4ad7cc15 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:11.692076 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1131/2277] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:46:11.702256 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed10da-accb-4b57-838a-20f1f99028fb None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:11.703563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d2ed10da-accb-4b57-838a-20f1f99028fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:11.724143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf7e20-5470-4f32-b6a2-affddc20422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:11.726753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf7e20-5470-4f32-b6a2-affddc20422c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.727036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf7e20-5470-4f32-b6a2-affddc20422c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.727424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf7e20-5470-4f32-b6a2-affddc20422c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.728138 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf7e20-5470-4f32-b6a2-affddc20422c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:11.730207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf7e20-5470-4f32-b6a2-affddc20422c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:11.757281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1147/2278] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:46:11 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:46:11.761254 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf7e20-5470-4f32-b6a2-affddc20422c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:11.763041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed10da-accb-4b57-838a-20f1f99028fb None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:11.766763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020692d5-93f6-48a2-9f01-aba1f9ed0376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:11.767496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020692d5-93f6-48a2-9f01-aba1f9ed0376 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.767836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020692d5-93f6-48a2-9f01-aba1f9ed0376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.768172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020692d5-93f6-48a2-9f01-aba1f9ed0376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.778290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020692d5-93f6-48a2-9f01-aba1f9ed0376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:11.780147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020692d5-93f6-48a2-9f01-aba1f9ed0376 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:11.781121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf7e20-5470-4f32-b6a2-affddc20422c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:11.785925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d184fb1-7c7a-4709-90b6-1c6168b540f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:11.786735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d184fb1-7c7a-4709-90b6-1c6168b540f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.787140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d184fb1-7c7a-4709-90b6-1c6168b540f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.787343 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d184fb1-7c7a-4709-90b6-1c6168b540f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.788136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d184fb1-7c7a-4709-90b6-1c6168b540f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:11.789778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d184fb1-7c7a-4709-90b6-1c6168b540f1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:11.792060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1132/2279] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:46:11.800528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1148/2280] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:11.804792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020692d5-93f6-48a2-9f01-aba1f9ed0376 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:11.814215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020692d5-93f6-48a2-9f01-aba1f9ed0376 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:11.819126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6e951-63ec-4409-acd8-da8c5d854aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:11.819787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6e951-63ec-4409-acd8-da8c5d854aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.820064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6e951-63ec-4409-acd8-da8c5d854aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.820333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6e951-63ec-4409-acd8-da8c5d854aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.821091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6e951-63ec-4409-acd8-da8c5d854aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:11.822730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6e951-63ec-4409-acd8-da8c5d854aa6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:11.833452 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1133/2281] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:11.837685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d184fb1-7c7a-4709-90b6-1c6168b540f1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:11.848241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d184fb1-7c7a-4709-90b6-1c6168b540f1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:11.853674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002acb1-875b-4203-bfa7-c3e376132d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:11.854850 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002acb1-875b-4203-bfa7-c3e376132d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.855178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002acb1-875b-4203-bfa7-c3e376132d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.855629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002acb1-875b-4203-bfa7-c3e376132d0e admin admin] PATH_INFO: `/v3/projects/8b93581c09a24bd1bef42446d3dbb097/users/87994c6ffd264f7fa2a859aa7f4bf39f/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.877430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002acb1-875b-4203-bfa7-c3e376132d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b93581c09a24bd1bef42446d3dbb097, user_id=87994c6ffd264f7fa2a859aa7f4bf39f, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:11.880259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002acb1-875b-4203-bfa7-c3e376132d0e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:11.896278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1149/2282] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:11 2022] PUT /identity/v3/projects/8b93581c09a24bd1bef42446d3dbb097/users/87994c6ffd264f7fa2a859aa7f4bf39f/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:11.906704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee3de3-9165-4d6b-a6cb-e1630e0b33a4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:11.906999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee3de3-9165-4d6b-a6cb-e1630e0b33a4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:11.907278 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee3de3-9165-4d6b-a6cb-e1630e0b33a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:11.934929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-03ee3de3-9165-4d6b-a6cb-e1630e0b33a4 None None] MFA Rules not processed for user `87994c6ffd264f7fa2a859aa7f4bf39f`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:11.936135 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-03ee3de3-9165-4d6b-a6cb-e1630e0b33a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:12.027380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1134/2283] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:12.039677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002acb1-875b-4203-bfa7-c3e376132d0e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.041743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6002acb1-875b-4203-bfa7-c3e376132d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:12.107409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee3de3-9165-4d6b-a6cb-e1630e0b33a4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.109039 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-03ee3de3-9165-4d6b-a6cb-e1630e0b33a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:12.111861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002acb1-875b-4203-bfa7-c3e376132d0e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.116187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eb014-056f-4b53-b539-f73fa7c5915a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:12.116630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eb014-056f-4b53-b539-f73fa7c5915a service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.116951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eb014-056f-4b53-b539-f73fa7c5915a service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.117230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eb014-056f-4b53-b539-f73fa7c5915a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.126326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261eb014-056f-4b53-b539-f73fa7c5915a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:12.127950 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261eb014-056f-4b53-b539-f73fa7c5915a service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:12.150864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1150/2284] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:46:12.180170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eb014-056f-4b53-b539-f73fa7c5915a service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.197613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee3de3-9165-4d6b-a6cb-e1630e0b33a4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.204205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:12.205205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.205439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.205711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] PATH_INFO: `/v3/projects/486da17bffa0460db801eb2b986576f2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.207049 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=486da17bffa0460db801eb2b986576f2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:12.209302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:12.248729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eb014-056f-4b53-b539-f73fa7c5915a service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.258278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:12.259611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.259881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.260163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] PATH_INFO: `/v3/users/87994c6ffd264f7fa2a859aa7f4bf39f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.261541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87994c6ffd264f7fa2a859aa7f4bf39f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:12.263507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:12.278292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486da17bffa0460db801eb2b986576f2', 'request_id': 'req-ef971704-fa47-482e-a6b1-b317a7df0636'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:12.288461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486da17bffa0460db801eb2b986576f2', 'request_id': 'req-ef971704-fa47-482e-a6b1-b317a7df0636'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:12.296429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486da17bffa0460db801eb2b986576f2', 'request_id': 'req-ef971704-fa47-482e-a6b1-b317a7df0636'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:12.309635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] The token cache is being invalidate because project 486da17bffa0460db801eb2b986576f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:46:12.309914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:12.310598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef971704-fa47-482e-a6b1-b317a7df0636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:12.312290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1135/2285] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:46:12 2022] DELETE /identity/v3/projects/486da17bffa0460db801eb2b986576f2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:46:12.314734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87994c6ffd264f7fa2a859aa7f4bf39f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:12.316269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87994c6ffd264f7fa2a859aa7f4bf39f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:12.318796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b5f7-2550-4fe1-886b-d00d1eb8459e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.319082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b5f7-2550-4fe1-886b-d00d1eb8459e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.319405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b5f7-2550-4fe1-886b-d00d1eb8459e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.331666 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87994c6ffd264f7fa2a859aa7f4bf39f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:12.340257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-30c3b5f7-2550-4fe1-886b-d00d1eb8459e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:12.341701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-30c3b5f7-2550-4fe1-886b-d00d1eb8459e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:12.342767 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87994c6ffd264f7fa2a859aa7f4bf39f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:12.350179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ec1f781e-03b5-4bc6-ac4b-9e9d86e4ff99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87994c6ffd264f7fa2a859aa7f4bf39f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:12.356173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1151/2286] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:46:12 2022] DELETE /identity/v3/users/87994c6ffd264f7fa2a859aa7f4bf39f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:12.360872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5db7c-dab4-47f5-808c-e4164b2cb9cf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.361133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5db7c-dab4-47f5-808c-e4164b2cb9cf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.361432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5db7c-dab4-47f5-808c-e4164b2cb9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.379600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d6e5db7c-dab4-47f5-808c-e4164b2cb9cf None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:12.381141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6e5db7c-dab4-47f5-808c-e4164b2cb9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:12.413311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1136/2287] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:46:12.421027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b75369-acd9-418b-a629-df481482e0e3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.421393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b75369-acd9-418b-a629-df481482e0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.421752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b75369-acd9-418b-a629-df481482e0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.442539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-28b75369-acd9-418b-a629-df481482e0e3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:12.444048 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-28b75369-acd9-418b-a629-df481482e0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:12.450877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1152/2288] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:46:12.453781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1137/2289] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:46:12.459772 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5db7c-dab4-47f5-808c-e4164b2cb9cf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.461696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b75369-acd9-418b-a629-df481482e0e3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.463246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-28b75369-acd9-418b-a629-df481482e0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:12.473441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5db7c-dab4-47f5-808c-e4164b2cb9cf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.480599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf196d9d-1bd9-428b-9380-d01aa7a5a236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:12.481379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf196d9d-1bd9-428b-9380-d01aa7a5a236 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.481726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf196d9d-1bd9-428b-9380-d01aa7a5a236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.482004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf196d9d-1bd9-428b-9380-d01aa7a5a236 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.482801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf196d9d-1bd9-428b-9380-d01aa7a5a236 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:46:12.483099 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf196d9d-1bd9-428b-9380-d01aa7a5a236 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:12.484849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf196d9d-1bd9-428b-9380-d01aa7a5a236 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:12.507948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1153/2290] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:46:12.507948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf196d9d-1bd9-428b-9380-d01aa7a5a236 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.517683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf196d9d-1bd9-428b-9380-d01aa7a5a236 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.522572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da668c0f-ac51-4079-8097-f719ca8188e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:12.523241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da668c0f-ac51-4079-8097-f719ca8188e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.523528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da668c0f-ac51-4079-8097-f719ca8188e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.523807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da668c0f-ac51-4079-8097-f719ca8188e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.524553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da668c0f-ac51-4079-8097-f719ca8188e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:12.526462 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da668c0f-ac51-4079-8097-f719ca8188e9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:12.554130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b75369-acd9-418b-a629-df481482e0e3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.558602 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1154/2291] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:46:12 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:12.559519 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da668c0f-ac51-4079-8097-f719ca8188e9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.615354 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341b53b-80f0-4877-8efd-0250de1297db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:12.616225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341b53b-80f0-4877-8efd-0250de1297db service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.616543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341b53b-80f0-4877-8efd-0250de1297db service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.616827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341b53b-80f0-4877-8efd-0250de1297db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.655202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da668c0f-ac51-4079-8097-f719ca8188e9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.660945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc650951-5444-4aa8-aae1-d30455e54b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:12.661659 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc650951-5444-4aa8-aae1-d30455e54b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.662000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc650951-5444-4aa8-aae1-d30455e54b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.662277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc650951-5444-4aa8-aae1-d30455e54b57 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.663180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc650951-5444-4aa8-aae1-d30455e54b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:12.665222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc650951-5444-4aa8-aae1-d30455e54b57 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:12.687449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1155/2292] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:46:12 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:12.692907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc650951-5444-4aa8-aae1-d30455e54b57 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.696108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341b53b-80f0-4877-8efd-0250de1297db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:12.697850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341b53b-80f0-4877-8efd-0250de1297db service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:12.703340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc650951-5444-4aa8-aae1-d30455e54b57 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.708028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1138/2293] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:46:12.708157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df5a78-8cec-410e-9341-b5e6f4a28fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:12.708962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df5a78-8cec-410e-9341-b5e6f4a28fef admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.709424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df5a78-8cec-410e-9341-b5e6f4a28fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.709769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df5a78-8cec-410e-9341-b5e6f4a28fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.710571 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df5a78-8cec-410e-9341-b5e6f4a28fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:12.712465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df5a78-8cec-410e-9341-b5e6f4a28fef admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:12.724530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1156/2294] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:46:12.727083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341b53b-80f0-4877-8efd-0250de1297db service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.738391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341b53b-80f0-4877-8efd-0250de1297db service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.746848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752c698-28f4-45d9-a866-73004f78b925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:12.747558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752c698-28f4-45d9-a866-73004f78b925 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.747839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752c698-28f4-45d9-a866-73004f78b925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.749718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752c698-28f4-45d9-a866-73004f78b925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.749718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752c698-28f4-45d9-a866-73004f78b925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:12.750535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752c698-28f4-45d9-a866-73004f78b925 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:12.768743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1139/2295] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:12.773968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df5a78-8cec-410e-9341-b5e6f4a28fef admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.785036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df5a78-8cec-410e-9341-b5e6f4a28fef admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.790585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7eb0a-83c4-43c1-a1c6-7fcd45337ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:12.791486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7eb0a-83c4-43c1-a1c6-7fcd45337ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.791771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7eb0a-83c4-43c1-a1c6-7fcd45337ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.792044 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7eb0a-83c4-43c1-a1c6-7fcd45337ed0 admin admin] PATH_INFO: `/v3/projects/8853dc3f41e147018cc50778b57b9c97/users/a1569db96d824c0e9f16b386df5e3413/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.809301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7eb0a-83c4-43c1-a1c6-7fcd45337ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8853dc3f41e147018cc50778b57b9c97, user_id=a1569db96d824c0e9f16b386df5e3413, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:12.811957 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7eb0a-83c4-43c1-a1c6-7fcd45337ed0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:12.826845 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1157/2296] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:12 2022] PUT /identity/v3/projects/8853dc3f41e147018cc50778b57b9c97/users/a1569db96d824c0e9f16b386df5e3413/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:12.831992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee581b-e8a1-4bc4-af15-0ed162cb19fc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:12.832325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee581b-e8a1-4bc4-af15-0ed162cb19fc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:12.832650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee581b-e8a1-4bc4-af15-0ed162cb19fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:12.850301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bfee581b-e8a1-4bc4-af15-0ed162cb19fc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:12.851651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bfee581b-e8a1-4bc4-af15-0ed162cb19fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:12.919543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1140/2297] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:46:12.930056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7eb0a-83c4-43c1-a1c6-7fcd45337ed0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:12.991642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7eb0a-83c4-43c1-a1c6-7fcd45337ed0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:12.997392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee581b-e8a1-4bc4-af15-0ed162cb19fc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:13.001993 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bfee581b-e8a1-4bc4-af15-0ed162cb19fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:13.008028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44307508-5a66-4055-a977-a404a584e15b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:13.011033 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44307508-5a66-4055-a977-a404a584e15b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.011485 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44307508-5a66-4055-a977-a404a584e15b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.011854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44307508-5a66-4055-a977-a404a584e15b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.026326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee581b-e8a1-4bc4-af15-0ed162cb19fc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:13.026682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44307508-5a66-4055-a977-a404a584e15b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:13.029341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44307508-5a66-4055-a977-a404a584e15b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:13.039554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:13.040445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.040568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1158/2298] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:46:13.041052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.041146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] PATH_INFO: `/v3/projects/8b93581c09a24bd1bef42446d3dbb097` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.042324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b93581c09a24bd1bef42446d3dbb097)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:13.044273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:13.097970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b93581c09a24bd1bef42446d3dbb097', 'request_id': 'req-2c0fca78-8ca6-444f-96c5-5834dac9c311'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:13.107353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b93581c09a24bd1bef42446d3dbb097', 'request_id': 'req-2c0fca78-8ca6-444f-96c5-5834dac9c311'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:13.114474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b93581c09a24bd1bef42446d3dbb097', 'request_id': 'req-2c0fca78-8ca6-444f-96c5-5834dac9c311'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:13.127142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] The token cache is being invalidate because project 8b93581c09a24bd1bef42446d3dbb097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:46:13.127336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:13.128239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2c0fca78-8ca6-444f-96c5-5834dac9c311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:13.130151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1141/2299] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:46:12 2022] DELETE /identity/v3/projects/8b93581c09a24bd1bef42446d3dbb097 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:13.135445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be362e9-f6b1-44ff-a1b0-991d30c4bf1f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.135601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be362e9-f6b1-44ff-a1b0-991d30c4bf1f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.136065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be362e9-f6b1-44ff-a1b0-991d30c4bf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.155164 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5be362e9-f6b1-44ff-a1b0-991d30c4bf1f None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:13.156368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5be362e9-f6b1-44ff-a1b0-991d30c4bf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:13.217168 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1159/2300] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:46:13.226194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd77db-2ea7-44b1-8bb8-6da9615f00a6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.226599 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd77db-2ea7-44b1-8bb8-6da9615f00a6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.226977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd77db-2ea7-44b1-8bb8-6da9615f00a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.245531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f0cd77db-2ea7-44b1-8bb8-6da9615f00a6 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:13.246751 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f0cd77db-2ea7-44b1-8bb8-6da9615f00a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:13.254869 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1142/2301] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:13.259682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be362e9-f6b1-44ff-a1b0-991d30c4bf1f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:13.271793 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be362e9-f6b1-44ff-a1b0-991d30c4bf1f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:13.281202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64fca6-2e74-4322-a6d6-aa31ca62df70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:13.282077 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64fca6-2e74-4322-a6d6-aa31ca62df70 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.282363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64fca6-2e74-4322-a6d6-aa31ca62df70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.283054 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64fca6-2e74-4322-a6d6-aa31ca62df70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.283699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64fca6-2e74-4322-a6d6-aa31ca62df70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:46:13.284147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64fca6-2e74-4322-a6d6-aa31ca62df70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:13.286198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64fca6-2e74-4322-a6d6-aa31ca62df70 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:13.303027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1160/2302] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:46:13.307597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd77db-2ea7-44b1-8bb8-6da9615f00a6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:13.317152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd77db-2ea7-44b1-8bb8-6da9615f00a6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:13.322771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2dc03-5b4f-4476-9380-ba797b3a819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:13.323470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2dc03-5b4f-4476-9380-ba797b3a819c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.323747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2dc03-5b4f-4476-9380-ba797b3a819c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.324045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2dc03-5b4f-4476-9380-ba797b3a819c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.325112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2dc03-5b4f-4476-9380-ba797b3a819c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:13.327045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2dc03-5b4f-4476-9380-ba797b3a819c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:13.356074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1143/2303] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:46:13 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:46:13.360471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64fca6-2e74-4322-a6d6-aa31ca62df70 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:13.374122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64fca6-2e74-4322-a6d6-aa31ca62df70 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:13.432004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9823bbf-6cab-45dc-b5a4-e107a84746d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:13.432702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9823bbf-6cab-45dc-b5a4-e107a84746d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.432984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9823bbf-6cab-45dc-b5a4-e107a84746d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.433255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9823bbf-6cab-45dc-b5a4-e107a84746d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.433966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9823bbf-6cab-45dc-b5a4-e107a84746d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:13.435879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9823bbf-6cab-45dc-b5a4-e107a84746d2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:13.461340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1161/2304] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:46:13 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:13.463169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2dc03-5b4f-4476-9380-ba797b3a819c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:13.473483 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2dc03-5b4f-4476-9380-ba797b3a819c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:13.478509 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc49eb6-9390-4c8f-838d-689597112dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:13.479183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc49eb6-9390-4c8f-838d-689597112dda admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.479465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc49eb6-9390-4c8f-838d-689597112dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.479738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc49eb6-9390-4c8f-838d-689597112dda admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.480508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc49eb6-9390-4c8f-838d-689597112dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:13.482213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc49eb6-9390-4c8f-838d-689597112dda admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:13.493235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1144/2305] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:13.497292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9823bbf-6cab-45dc-b5a4-e107a84746d2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:13.508095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9823bbf-6cab-45dc-b5a4-e107a84746d2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:13.513306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fd3f8-1d62-4f74-a881-a7a8acedb6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:13.513979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fd3f8-1d62-4f74-a881-a7a8acedb6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.514257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fd3f8-1d62-4f74-a881-a7a8acedb6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.514526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fd3f8-1d62-4f74-a881-a7a8acedb6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.515249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172fd3f8-1d62-4f74-a881-a7a8acedb6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:13.516936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172fd3f8-1d62-4f74-a881-a7a8acedb6c0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:13.527770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1162/2306] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:46:13.532050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc49eb6-9390-4c8f-838d-689597112dda admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:13.546236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc49eb6-9390-4c8f-838d-689597112dda admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:13.550989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32e26a-7f98-4ce4-84aa-9cebe79dbd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:13.551921 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32e26a-7f98-4ce4-84aa-9cebe79dbd62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.552218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32e26a-7f98-4ce4-84aa-9cebe79dbd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.552590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32e26a-7f98-4ce4-84aa-9cebe79dbd62 admin admin] PATH_INFO: `/v3/projects/bc8b9f6c3c8844e483955fde21ea0db9/users/cbd9a26afe174b09adb5e11aeb1a67a9/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.569294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32e26a-7f98-4ce4-84aa-9cebe79dbd62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8b9f6c3c8844e483955fde21ea0db9, user_id=cbd9a26afe174b09adb5e11aeb1a67a9, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:13.571938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32e26a-7f98-4ce4-84aa-9cebe79dbd62 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:13.585440 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1145/2307] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:13 2022] PUT /identity/v3/projects/bc8b9f6c3c8844e483955fde21ea0db9/users/cbd9a26afe174b09adb5e11aeb1a67a9/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:13.595785 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668a02b-43fa-4cb2-9083-9b6a8ddc0907 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.596142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668a02b-43fa-4cb2-9083-9b6a8ddc0907 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.596901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668a02b-43fa-4cb2-9083-9b6a8ddc0907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.639796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7668a02b-43fa-4cb2-9083-9b6a8ddc0907 None None] MFA Rules not processed for user `cbd9a26afe174b09adb5e11aeb1a67a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:13.641033 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7668a02b-43fa-4cb2-9083-9b6a8ddc0907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:13.714087 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1163/2308] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:46:13.726490 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32e26a-7f98-4ce4-84aa-9cebe79dbd62 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:13.728924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4d32e26a-7f98-4ce4-84aa-9cebe79dbd62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:13.814322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32e26a-7f98-4ce4-84aa-9cebe79dbd62 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:13.819656 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6de246-cae3-44bb-ac4f-e2e4ea305a01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:13.820534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6de246-cae3-44bb-ac4f-e2e4ea305a01 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:13.820908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6de246-cae3-44bb-ac4f-e2e4ea305a01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:13.821304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6de246-cae3-44bb-ac4f-e2e4ea305a01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:13.834918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6de246-cae3-44bb-ac4f-e2e4ea305a01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:13.836639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6de246-cae3-44bb-ac4f-e2e4ea305a01 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:13.847851 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1146/2309] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:46:16.377412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e02e3-d32e-4862-9710-894c1a7e54e1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:16.377879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e02e3-d32e-4862-9710-894c1a7e54e1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:16.377995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e02e3-d32e-4862-9710-894c1a7e54e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:16.410109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-213e02e3-d32e-4862-9710-894c1a7e54e1 None None] MFA Rules not processed for user `cbd9a26afe174b09adb5e11aeb1a67a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:16.411452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-213e02e3-d32e-4862-9710-894c1a7e54e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:16.424268 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1164/2310] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:17.591535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cc31e-e4ff-4e7e-8840-2bee1858bba0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:17.591535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cc31e-e4ff-4e7e-8840-2bee1858bba0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:17.591535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cc31e-e4ff-4e7e-8840-2bee1858bba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:17.623842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e71cc31e-e4ff-4e7e-8840-2bee1858bba0 None None] MFA Rules not processed for user `a1569db96d824c0e9f16b386df5e3413`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:17.625357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e71cc31e-e4ff-4e7e-8840-2bee1858bba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:17.702355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1147/2311] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:17.706887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e02e3-d32e-4862-9710-894c1a7e54e1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:17.709451 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-213e02e3-d32e-4862-9710-894c1a7e54e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:17.789544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e02e3-d32e-4862-9710-894c1a7e54e1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:17.794764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a00574-53c6-425b-a254-d5cf93cf9f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:17.795434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00574-53c6-425b-a254-d5cf93cf9f5f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:17.795720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00574-53c6-425b-a254-d5cf93cf9f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:17.796113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00574-53c6-425b-a254-d5cf93cf9f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:17.796919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a00574-53c6-425b-a254-d5cf93cf9f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:17.799098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a00574-53c6-425b-a254-d5cf93cf9f5f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:17.831285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1165/2312] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:46:17 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:17.835416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cc31e-e4ff-4e7e-8840-2bee1858bba0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:17.846357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cc31e-e4ff-4e7e-8840-2bee1858bba0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:17.903328 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ba8a3-8ff8-4d2b-8c9a-4c47ba4bcfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:17.904012 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ba8a3-8ff8-4d2b-8c9a-4c47ba4bcfdb admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:17.904292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ba8a3-8ff8-4d2b-8c9a-4c47ba4bcfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:17.904595 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ba8a3-8ff8-4d2b-8c9a-4c47ba4bcfdb admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:17.905303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ba8a3-8ff8-4d2b-8c9a-4c47ba4bcfdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:17.907235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ba8a3-8ff8-4d2b-8c9a-4c47ba4bcfdb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:17.930631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1148/2313] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:46:17 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:17.936370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a00574-53c6-425b-a254-d5cf93cf9f5f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:17.950636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a00574-53c6-425b-a254-d5cf93cf9f5f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:17.956264 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1301420-01f1-448f-aca0-4cac460d4d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:17.957083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1301420-01f1-448f-aca0-4cac460d4d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:17.957241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1301420-01f1-448f-aca0-4cac460d4d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:17.957518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1301420-01f1-448f-aca0-4cac460d4d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:17.958278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1301420-01f1-448f-aca0-4cac460d4d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:17.959975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1301420-01f1-448f-aca0-4cac460d4d7d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:17.972232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1166/2314] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:17.977014 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ba8a3-8ff8-4d2b-8c9a-4c47ba4bcfdb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:17.988275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ba8a3-8ff8-4d2b-8c9a-4c47ba4bcfdb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:17.994567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c4e12-5410-42d1-803d-f095e4f484e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:17.995513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c4e12-5410-42d1-803d-f095e4f484e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:17.995792 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c4e12-5410-42d1-803d-f095e4f484e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:17.996079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c4e12-5410-42d1-803d-f095e4f484e6 admin admin] PATH_INFO: `/v3/projects/c8229262ea9a40e4970b89e8adadc884/users/4648aa4adcfc480b968b9b2abd2d7b17/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:18.015593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c4e12-5410-42d1-803d-f095e4f484e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8229262ea9a40e4970b89e8adadc884, user_id=4648aa4adcfc480b968b9b2abd2d7b17, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:18.019777 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c4e12-5410-42d1-803d-f095e4f484e6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:18.036422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1149/2315] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:17 2022] PUT /identity/v3/projects/c8229262ea9a40e4970b89e8adadc884/users/4648aa4adcfc480b968b9b2abd2d7b17/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:18.040532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1301420-01f1-448f-aca0-4cac460d4d7d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:18.052099 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1301420-01f1-448f-aca0-4cac460d4d7d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:18.117034 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf94b77-fbae-4411-baaf-9afa00a4bd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:18.117655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf94b77-fbae-4411-baaf-9afa00a4bd27 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:18.117933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf94b77-fbae-4411-baaf-9afa00a4bd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:18.118207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf94b77-fbae-4411-baaf-9afa00a4bd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:18.118945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf94b77-fbae-4411-baaf-9afa00a4bd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:18.120621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf94b77-fbae-4411-baaf-9afa00a4bd27 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:18.130801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1167/2316] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:18.135260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c4e12-5410-42d1-803d-f095e4f484e6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:18.146330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c4e12-5410-42d1-803d-f095e4f484e6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:18.152383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1861e-cadf-4d1a-9b8d-98c5e7af0264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:18.153387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1861e-cadf-4d1a-9b8d-98c5e7af0264 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:18.153664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1861e-cadf-4d1a-9b8d-98c5e7af0264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:18.153936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1861e-cadf-4d1a-9b8d-98c5e7af0264 admin admin] PATH_INFO: `/v3/projects/c8229262ea9a40e4970b89e8adadc884/users/4648aa4adcfc480b968b9b2abd2d7b17/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:18.156262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1861e-cadf-4d1a-9b8d-98c5e7af0264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8229262ea9a40e4970b89e8adadc884, user_id=4648aa4adcfc480b968b9b2abd2d7b17, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:18.158892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1861e-cadf-4d1a-9b8d-98c5e7af0264 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:18.170859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1150/2317] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:18 2022] PUT /identity/v3/projects/c8229262ea9a40e4970b89e8adadc884/users/4648aa4adcfc480b968b9b2abd2d7b17/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:23.770683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38614efc-c1d5-4ef0-925a-1cc813bd3011 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:23.772314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38614efc-c1d5-4ef0-925a-1cc813bd3011 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:23.772630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38614efc-c1d5-4ef0-925a-1cc813bd3011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:23.809877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-38614efc-c1d5-4ef0-925a-1cc813bd3011 None None] MFA Rules not processed for user `4648aa4adcfc480b968b9b2abd2d7b17`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:23.811667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-38614efc-c1d5-4ef0-925a-1cc813bd3011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:23.925847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1861e-cadf-4d1a-9b8d-98c5e7af0264 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:23.929313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1168/2318] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:23.932124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4fd1861e-cadf-4d1a-9b8d-98c5e7af0264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:24.048095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1861e-cadf-4d1a-9b8d-98c5e7af0264 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:24.059358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59125f29-9326-4fb3-880c-ace4ec54164c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:24.060616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59125f29-9326-4fb3-880c-ace4ec54164c service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:24.060901 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59125f29-9326-4fb3-880c-ace4ec54164c service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:24.061173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59125f29-9326-4fb3-880c-ace4ec54164c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:24.152779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59125f29-9326-4fb3-880c-ace4ec54164c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:24.154776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59125f29-9326-4fb3-880c-ace4ec54164c service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:24.179106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1151/2319] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:46:40.004576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38614efc-c1d5-4ef0-925a-1cc813bd3011 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:40.006908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-38614efc-c1d5-4ef0-925a-1cc813bd3011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:40.102012 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38614efc-c1d5-4ef0-925a-1cc813bd3011 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:40.107485 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355e773-e031-4129-a171-d0a9ce9442ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e934c8e7dd0425cac68eb91e5e2c0ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:40.108142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355e773-e031-4129-a171-d0a9ce9442ed service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:40.108425 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355e773-e031-4129-a171-d0a9ce9442ed service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:40.108727 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355e773-e031-4129-a171-d0a9ce9442ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:40.110506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1355e773-e031-4129-a171-d0a9ce9442ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:40.122166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355e773-e031-4129-a171-d0a9ce9442ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:40.123746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355e773-e031-4129-a171-d0a9ce9442ed service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:40.133482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1169/2320] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:46:51.589298 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7808a-644a-4daf-85d7-15ed442a6358 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:51.589767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7808a-644a-4daf-85d7-15ed442a6358 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:51.589942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7808a-644a-4daf-85d7-15ed442a6358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:51.651670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-23b7808a-644a-4daf-85d7-15ed442a6358 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:51.652898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-23b7808a-644a-4daf-85d7-15ed442a6358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:51.742231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1152/2321] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:46:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:51.809298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355e773-e031-4129-a171-d0a9ce9442ed service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:51.811028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1355e773-e031-4129-a171-d0a9ce9442ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:51.928801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355e773-e031-4129-a171-d0a9ce9442ed service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:51.935815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c258f2f-cf30-41d8-8f62-fec30c5bb140 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:51.936539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c258f2f-cf30-41d8-8f62-fec30c5bb140 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:51.936851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c258f2f-cf30-41d8-8f62-fec30c5bb140 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:51.937147 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c258f2f-cf30-41d8-8f62-fec30c5bb140 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:51.945885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c258f2f-cf30-41d8-8f62-fec30c5bb140 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:51.947570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c258f2f-cf30-41d8-8f62-fec30c5bb140 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:51.958084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1170/2322] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:46:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:46:52.630310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee06179-f7d1-4bd3-a7b0-861f050306ee None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:52.630692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee06179-f7d1-4bd3-a7b0-861f050306ee None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:52.630858 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee06179-f7d1-4bd3-a7b0-861f050306ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:52.648075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1ee06179-f7d1-4bd3-a7b0-861f050306ee None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:52.649263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1ee06179-f7d1-4bd3-a7b0-861f050306ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:52.656636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1153/2323] 10.222.0.50 () {58 vars in 967 bytes} [Fri Apr 29 18:46:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:52.670502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c258f2f-cf30-41d8-8f62-fec30c5bb140 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:52.681925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c258f2f-cf30-41d8-8f62-fec30c5bb140 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:52.687694 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96485e0-b9f4-4ce7-a9a0-de5ac79d5e2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:52.688423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96485e0-b9f4-4ce7-a9a0-de5ac79d5e2e service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:52.688901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96485e0-b9f4-4ce7-a9a0-de5ac79d5e2e service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:52.689173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96485e0-b9f4-4ce7-a9a0-de5ac79d5e2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:52.697416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96485e0-b9f4-4ce7-a9a0-de5ac79d5e2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:52.699360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96485e0-b9f4-4ce7-a9a0-de5ac79d5e2e service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:52.707496 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1171/2324] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:46:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:46:52.935635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edbbbd-f2be-48ef-8f9b-b9ad3f365fc4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:52.935927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edbbbd-f2be-48ef-8f9b-b9ad3f365fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:52.936211 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edbbbd-f2be-48ef-8f9b-b9ad3f365fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:52.955239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e9edbbbd-f2be-48ef-8f9b-b9ad3f365fc4 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:52.956446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e9edbbbd-f2be-48ef-8f9b-b9ad3f365fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:52.964507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1154/2325] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:46:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:52.976885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96485e0-b9f4-4ce7-a9a0-de5ac79d5e2e service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:52.988848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96485e0-b9f4-4ce7-a9a0-de5ac79d5e2e service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:52.994358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e300d9-82a7-4ff4-ba3a-67e72cf21436 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:52.995004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e300d9-82a7-4ff4-ba3a-67e72cf21436 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:52.995274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e300d9-82a7-4ff4-ba3a-67e72cf21436 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:52.995564 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e300d9-82a7-4ff4-ba3a-67e72cf21436 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:53.003965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e300d9-82a7-4ff4-ba3a-67e72cf21436 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:53.005579 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e300d9-82a7-4ff4-ba3a-67e72cf21436 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:53.013694 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1172/2326] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:46:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:46:53.345053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c01010-0d87-4939-bb6b-5ff1c5531018 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:53.345429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c01010-0d87-4939-bb6b-5ff1c5531018 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:53.345851 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c01010-0d87-4939-bb6b-5ff1c5531018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:53.364302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-06c01010-0d87-4939-bb6b-5ff1c5531018 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:53.365570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-06c01010-0d87-4939-bb6b-5ff1c5531018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:53.373115 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1155/2327] 10.222.0.50 () {58 vars in 967 bytes} [Fri Apr 29 18:46:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:53.386036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e300d9-82a7-4ff4-ba3a-67e72cf21436 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:53.396718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e300d9-82a7-4ff4-ba3a-67e72cf21436 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:53.401762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d93a2-d9f2-435e-a6b5-79c0345c1cf4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:53.402405 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d93a2-d9f2-435e-a6b5-79c0345c1cf4 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:53.402687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d93a2-d9f2-435e-a6b5-79c0345c1cf4 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:53.402962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d93a2-d9f2-435e-a6b5-79c0345c1cf4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:53.411255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d93a2-d9f2-435e-a6b5-79c0345c1cf4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:53.412864 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d93a2-d9f2-435e-a6b5-79c0345c1cf4 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:53.420969 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1173/2328] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:46:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:46:55.504559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c01010-0d87-4939-bb6b-5ff1c5531018 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:55.573905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c01010-0d87-4939-bb6b-5ff1c5531018 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:55.579009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:55.580120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:55.580399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:55.580697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] PATH_INFO: `/v3/users/cbd9a26afe174b09adb5e11aeb1a67a9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:55.581995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd9a26afe174b09adb5e11aeb1a67a9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:55.584038 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:55.641811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd9a26afe174b09adb5e11aeb1a67a9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:55.642705 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd9a26afe174b09adb5e11aeb1a67a9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:55.655891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd9a26afe174b09adb5e11aeb1a67a9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:55.663562 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd9a26afe174b09adb5e11aeb1a67a9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:55.672844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd9a26afe174b09adb5e11aeb1a67a9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:55.679079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1156/2329] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:46:55 2022] DELETE /identity/v3/users/cbd9a26afe174b09adb5e11aeb1a67a9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:55.683741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2bf00-b9ee-40f4-a3cb-37d01fd07862 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:55.684548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2bf00-b9ee-40f4-a3cb-37d01fd07862 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:55.684975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2bf00-b9ee-40f4-a3cb-37d01fd07862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:55.702873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6ee2bf00-b9ee-40f4-a3cb-37d01fd07862 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:55.704077 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ee2bf00-b9ee-40f4-a3cb-37d01fd07862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:55.766945 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1174/2330] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:55.778707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:55.780961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:55.851329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cfb98-f51f-4ce6-9552-810010faebb5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:55.856078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6366b-ca2e-4d60-bdc8-7f4434fd713f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:55.856764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6366b-ca2e-4d60-bdc8-7f4434fd713f service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:55.857008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6366b-ca2e-4d60-bdc8-7f4434fd713f service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:55.857297 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6366b-ca2e-4d60-bdc8-7f4434fd713f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:55.867923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d6366b-ca2e-4d60-bdc8-7f4434fd713f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:55.869342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d6366b-ca2e-4d60-bdc8-7f4434fd713f service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:55.879389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1157/2331] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:46:56.058920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2bf00-b9ee-40f4-a3cb-37d01fd07862 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:56.060621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ee2bf00-b9ee-40f4-a3cb-37d01fd07862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:56.074446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2bf00-b9ee-40f4-a3cb-37d01fd07862 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:56.080185 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:56.081261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.081667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.082104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] PATH_INFO: `/v3/projects/bc8b9f6c3c8844e483955fde21ea0db9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.083320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8b9f6c3c8844e483955fde21ea0db9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:56.085475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:56.132035 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8b9f6c3c8844e483955fde21ea0db9', 'request_id': 'req-d398a57e-b3d7-4239-99f1-bac0fcac52b4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:56.140821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8b9f6c3c8844e483955fde21ea0db9', 'request_id': 'req-d398a57e-b3d7-4239-99f1-bac0fcac52b4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:56.147277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8b9f6c3c8844e483955fde21ea0db9', 'request_id': 'req-d398a57e-b3d7-4239-99f1-bac0fcac52b4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:56.157254 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] The token cache is being invalidate because project bc8b9f6c3c8844e483955fde21ea0db9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:46:56.157537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:56.158275 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d398a57e-b3d7-4239-99f1-bac0fcac52b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:46:56.159821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1175/2332] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:46:56 2022] DELETE /identity/v3/projects/bc8b9f6c3c8844e483955fde21ea0db9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:56.165057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab50e7e-e320-4b10-9d91-6563e1e5b248 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.165345 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab50e7e-e320-4b10-9d91-6563e1e5b248 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.165641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab50e7e-e320-4b10-9d91-6563e1e5b248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.183575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3ab50e7e-e320-4b10-9d91-6563e1e5b248 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:56.184740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3ab50e7e-e320-4b10-9d91-6563e1e5b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:56.248827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1158/2333] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:56.260035 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d99ce-bb55-42ee-898a-3ff1a68f1315 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.260221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d99ce-bb55-42ee-898a-3ff1a68f1315 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.260543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d99ce-bb55-42ee-898a-3ff1a68f1315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.280379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-981d99ce-bb55-42ee-898a-3ff1a68f1315 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:56.281629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-981d99ce-bb55-42ee-898a-3ff1a68f1315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:56.291374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1176/2334] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:46:56.295771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab50e7e-e320-4b10-9d91-6563e1e5b248 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:56.306032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab50e7e-e320-4b10-9d91-6563e1e5b248 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:56.311964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b75b5c-6637-419f-8dd8-08084ed94613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:56.312656 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b75b5c-6637-419f-8dd8-08084ed94613 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.312940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b75b5c-6637-419f-8dd8-08084ed94613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.313232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b75b5c-6637-419f-8dd8-08084ed94613 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.314006 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b75b5c-6637-419f-8dd8-08084ed94613 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:46:56.314717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b75b5c-6637-419f-8dd8-08084ed94613 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:56.316231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b75b5c-6637-419f-8dd8-08084ed94613 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:56.329318 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1159/2335] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:46:56.334045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d99ce-bb55-42ee-898a-3ff1a68f1315 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:56.343743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d99ce-bb55-42ee-898a-3ff1a68f1315 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:56.349690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd266d-5ccc-4893-8d90-b89fcfe90374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:56.350339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd266d-5ccc-4893-8d90-b89fcfe90374 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.350616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd266d-5ccc-4893-8d90-b89fcfe90374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.350885 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd266d-5ccc-4893-8d90-b89fcfe90374 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.351590 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd266d-5ccc-4893-8d90-b89fcfe90374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:56.353555 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd266d-5ccc-4893-8d90-b89fcfe90374 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:56.380120 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1177/2336] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:46:56 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:56.384355 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b75b5c-6637-419f-8dd8-08084ed94613 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:56.394803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b75b5c-6637-419f-8dd8-08084ed94613 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:56.450475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2461883-56ce-4ecb-b4ac-2ef21feb1b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:56.451168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2461883-56ce-4ecb-b4ac-2ef21feb1b0f admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.451410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2461883-56ce-4ecb-b4ac-2ef21feb1b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.451696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2461883-56ce-4ecb-b4ac-2ef21feb1b0f admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.452373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2461883-56ce-4ecb-b4ac-2ef21feb1b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:56.454327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2461883-56ce-4ecb-b4ac-2ef21feb1b0f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:56.478492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1160/2337] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:46:56 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:46:56.479783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd266d-5ccc-4893-8d90-b89fcfe90374 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:56.490109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd266d-5ccc-4893-8d90-b89fcfe90374 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:56.495060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce591cf-91dc-4e61-aa55-e0d15c6125db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:56.495726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce591cf-91dc-4e61-aa55-e0d15c6125db admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.496005 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce591cf-91dc-4e61-aa55-e0d15c6125db admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.496276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce591cf-91dc-4e61-aa55-e0d15c6125db admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.497065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce591cf-91dc-4e61-aa55-e0d15c6125db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:56.498659 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce591cf-91dc-4e61-aa55-e0d15c6125db admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:56.511130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1178/2338] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:56.514928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2461883-56ce-4ecb-b4ac-2ef21feb1b0f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:56.525144 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2461883-56ce-4ecb-b4ac-2ef21feb1b0f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:56.530005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec485cd5-b548-423e-afc8-cdbbc1bfbafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:56.530730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec485cd5-b548-423e-afc8-cdbbc1bfbafd admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.531011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec485cd5-b548-423e-afc8-cdbbc1bfbafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.531278 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec485cd5-b548-423e-afc8-cdbbc1bfbafd admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.532032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec485cd5-b548-423e-afc8-cdbbc1bfbafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:56.534164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec485cd5-b548-423e-afc8-cdbbc1bfbafd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:56.547444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1161/2339] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:46:56.549178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce591cf-91dc-4e61-aa55-e0d15c6125db admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:56.558886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce591cf-91dc-4e61-aa55-e0d15c6125db admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:56.563994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944425a-a8e2-416f-bb82-b759fba14daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:56.565009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944425a-a8e2-416f-bb82-b759fba14daf admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.565287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944425a-a8e2-416f-bb82-b759fba14daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.565555 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944425a-a8e2-416f-bb82-b759fba14daf admin admin] PATH_INFO: `/v3/projects/ec05369f752b4bf1b68836cbb489afd7/users/ae59d6fc3f684b5ba08bb254d40e77db/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.580478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944425a-a8e2-416f-bb82-b759fba14daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec05369f752b4bf1b68836cbb489afd7, user_id=ae59d6fc3f684b5ba08bb254d40e77db, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:56.583104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944425a-a8e2-416f-bb82-b759fba14daf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:56.598079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1179/2340] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:46:56 2022] PUT /identity/v3/projects/ec05369f752b4bf1b68836cbb489afd7/users/ae59d6fc3f684b5ba08bb254d40e77db/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:46:56.608543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab060e-2431-4050-8a3c-313218dbfde2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.608882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab060e-2431-4050-8a3c-313218dbfde2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.609158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab060e-2431-4050-8a3c-313218dbfde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.637575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c3ab060e-2431-4050-8a3c-313218dbfde2 None None] MFA Rules not processed for user `ae59d6fc3f684b5ba08bb254d40e77db`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:46:56.638769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c3ab060e-2431-4050-8a3c-313218dbfde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:56.709002 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1162/2341] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:46:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:46:56.721223 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944425a-a8e2-416f-bb82-b759fba14daf admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:46:56.722677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a944425a-a8e2-416f-bb82-b759fba14daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:46:56.780876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944425a-a8e2-416f-bb82-b759fba14daf admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:46:56.785912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e74796-4d77-45e4-87f6-48dd0fac4b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:46:56.786681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e74796-4d77-45e4-87f6-48dd0fac4b4d service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:46:56.787474 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e74796-4d77-45e4-87f6-48dd0fac4b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:46:56.787474 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e74796-4d77-45e4-87f6-48dd0fac4b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:46:56.797649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e74796-4d77-45e4-87f6-48dd0fac4b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:46:56.799827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e74796-4d77-45e4-87f6-48dd0fac4b4d service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:46:56.812004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1180/2342] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:46:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:47:09.631364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3df81-7f80-4ed3-aec6-d8c3ac07869b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:09.631790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3df81-7f80-4ed3-aec6-d8c3ac07869b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:09.631949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3df81-7f80-4ed3-aec6-d8c3ac07869b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:09.652797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-07c3df81-7f80-4ed3-aec6-d8c3ac07869b None None] MFA Rules not processed for user `e9a2443e71c54075823f75fdb8df469e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:09.654394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-07c3df81-7f80-4ed3-aec6-d8c3ac07869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:09.716639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1163/2343] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:47:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:47:09.723962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd58f-e4b1-4ffa-b6b1-4478ad53e0e8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:09.724920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd58f-e4b1-4ffa-b6b1-4478ad53e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:09.725409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fd58f-e4b1-4ffa-b6b1-4478ad53e0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:09.745260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-518fd58f-e4b1-4ffa-b6b1-4478ad53e0e8 None None] MFA Rules not processed for user `a1569db96d824c0e9f16b386df5e3413`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:09.746419 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-518fd58f-e4b1-4ffa-b6b1-4478ad53e0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:09.830851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1181/2344] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:47:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:47:10.068259 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3df81-7f80-4ed3-aec6-d8c3ac07869b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:10.135482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3df81-7f80-4ed3-aec6-d8c3ac07869b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:10.140298 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:10.141225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:10.141508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:10.141779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] PATH_INFO: `/v3/users/ae59d6fc3f684b5ba08bb254d40e77db` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:10.142968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae59d6fc3f684b5ba08bb254d40e77db)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:10.144980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:10.196768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae59d6fc3f684b5ba08bb254d40e77db'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:47:10.198023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae59d6fc3f684b5ba08bb254d40e77db'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:47:10.210380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae59d6fc3f684b5ba08bb254d40e77db'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:47:10.217427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae59d6fc3f684b5ba08bb254d40e77db'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:47:10.228056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae59d6fc3f684b5ba08bb254d40e77db'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:47:10.234285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1164/2345] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:47:10 2022] DELETE /identity/v3/users/ae59d6fc3f684b5ba08bb254d40e77db => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:47:10.238841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5deae-5319-4819-93b2-471ba33b51f0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:10.239276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5deae-5319-4819-93b2-471ba33b51f0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:10.240241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5deae-5319-4819-93b2-471ba33b51f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:10.260010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e5d5deae-5319-4819-93b2-471ba33b51f0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:10.261163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e5d5deae-5319-4819-93b2-471ba33b51f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:10.337156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1182/2346] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:47:10.348884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:10.350872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:10.422030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e55d7f-b058-40c8-ae73-11e41fb4585c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:10.426090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab7fbc-82b6-4e63-be70-e78cb32b3b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:10.426756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab7fbc-82b6-4e63-be70-e78cb32b3b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:10.427167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab7fbc-82b6-4e63-be70-e78cb32b3b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:10.427597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab7fbc-82b6-4e63-be70-e78cb32b3b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:10.437056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab7fbc-82b6-4e63-be70-e78cb32b3b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:10.438494 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab7fbc-82b6-4e63-be70-e78cb32b3b4a service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:10.452003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1165/2347] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:47:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:47:10.905662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5deae-5319-4819-93b2-471ba33b51f0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:10.908000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e5d5deae-5319-4819-93b2-471ba33b51f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:10.925196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5deae-5319-4819-93b2-471ba33b51f0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:10.931670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:10.932591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:10.932912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:10.933143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] PATH_INFO: `/v3/projects/ec05369f752b4bf1b68836cbb489afd7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:10.934431 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec05369f752b4bf1b68836cbb489afd7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:10.938709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:10.993272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec05369f752b4bf1b68836cbb489afd7', 'request_id': 'req-0dac4048-7a65-42e0-a4ea-a15736183c45'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:47:11.007574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec05369f752b4bf1b68836cbb489afd7', 'request_id': 'req-0dac4048-7a65-42e0-a4ea-a15736183c45'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:47:11.015820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec05369f752b4bf1b68836cbb489afd7', 'request_id': 'req-0dac4048-7a65-42e0-a4ea-a15736183c45'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:47:11.027269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] The token cache is being invalidate because project ec05369f752b4bf1b68836cbb489afd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:47:11.027556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:47:11.028375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0dac4048-7a65-42e0-a4ea-a15736183c45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:47:11.030035 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1183/2348] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:47:10 2022] DELETE /identity/v3/projects/ec05369f752b4bf1b68836cbb489afd7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:47:11.035430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b680c-02a5-49c9-bb18-3561be765d4a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:11.035986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b680c-02a5-49c9-bb18-3561be765d4a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:11.035986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b680c-02a5-49c9-bb18-3561be765d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:11.055117 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e82b680c-02a5-49c9-bb18-3561be765d4a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:11.056549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e82b680c-02a5-49c9-bb18-3561be765d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:11.121081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1166/2349] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:47:11.128504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca5576-993a-4faf-b009-8b2f389a5504 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:11.128874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca5576-993a-4faf-b009-8b2f389a5504 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:11.129246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca5576-993a-4faf-b009-8b2f389a5504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:11.147669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d9ca5576-993a-4faf-b009-8b2f389a5504 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:11.148926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d9ca5576-993a-4faf-b009-8b2f389a5504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:11.156809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1184/2350] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:47:11.161145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b680c-02a5-49c9-bb18-3561be765d4a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:11.170877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b680c-02a5-49c9-bb18-3561be765d4a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:11.175329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2650d-392e-414e-98c2-7157f06b5e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:11.175984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2650d-392e-414e-98c2-7157f06b5e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:11.176273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2650d-392e-414e-98c2-7157f06b5e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:11.176552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2650d-392e-414e-98c2-7157f06b5e9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:11.177446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2650d-392e-414e-98c2-7157f06b5e9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:47:11.178015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2650d-392e-414e-98c2-7157f06b5e9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:11.179629 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2650d-392e-414e-98c2-7157f06b5e9d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:11.191388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1167/2351] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:47:11.196084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca5576-993a-4faf-b009-8b2f389a5504 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:11.205696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca5576-993a-4faf-b009-8b2f389a5504 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:11.210737 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc10b0c-64a4-4c54-a120-17f8a8ef27b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:11.211397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc10b0c-64a4-4c54-a120-17f8a8ef27b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:11.211705 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc10b0c-64a4-4c54-a120-17f8a8ef27b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:11.211953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc10b0c-64a4-4c54-a120-17f8a8ef27b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:11.212686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc10b0c-64a4-4c54-a120-17f8a8ef27b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:11.214588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc10b0c-64a4-4c54-a120-17f8a8ef27b8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:11.240288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1185/2352] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:47:11 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:47:11.244423 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2650d-392e-414e-98c2-7157f06b5e9d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:11.254096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2650d-392e-414e-98c2-7157f06b5e9d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:11.310549 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374530c-a870-40ff-88de-2bdc119685f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:11.311120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374530c-a870-40ff-88de-2bdc119685f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:11.311399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374530c-a870-40ff-88de-2bdc119685f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:11.311668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374530c-a870-40ff-88de-2bdc119685f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:11.313119 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374530c-a870-40ff-88de-2bdc119685f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:11.315406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374530c-a870-40ff-88de-2bdc119685f3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:11.336313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1168/2353] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:47:11 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:47:11.340466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc10b0c-64a4-4c54-a120-17f8a8ef27b8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:11.350791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc10b0c-64a4-4c54-a120-17f8a8ef27b8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:11.355418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77f2c-1076-4983-ae55-7545da802c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:11.356073 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77f2c-1076-4983-ae55-7545da802c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:11.356356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77f2c-1076-4983-ae55-7545da802c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:11.356659 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77f2c-1076-4983-ae55-7545da802c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:11.357377 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c77f2c-1076-4983-ae55-7545da802c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:11.358975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c77f2c-1076-4983-ae55-7545da802c26 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:11.369274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1186/2354] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:47:11.373193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374530c-a870-40ff-88de-2bdc119685f3 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:11.383692 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374530c-a870-40ff-88de-2bdc119685f3 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:11.389362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e8a15-72c1-4304-90ad-77dab54fb914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:11.390040 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e8a15-72c1-4304-90ad-77dab54fb914 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:11.390314 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e8a15-72c1-4304-90ad-77dab54fb914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:11.390586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e8a15-72c1-4304-90ad-77dab54fb914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:11.391314 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e8a15-72c1-4304-90ad-77dab54fb914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:11.392969 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e8a15-72c1-4304-90ad-77dab54fb914 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:11.403223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1169/2355] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:47:11.407471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77f2c-1076-4983-ae55-7545da802c26 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:11.417861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77f2c-1076-4983-ae55-7545da802c26 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:11.423807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de53e4d-602a-481c-8ddf-6eb6f977b84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:11.424734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de53e4d-602a-481c-8ddf-6eb6f977b84b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:11.425015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de53e4d-602a-481c-8ddf-6eb6f977b84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:11.425286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de53e4d-602a-481c-8ddf-6eb6f977b84b admin admin] PATH_INFO: `/v3/projects/cbeb3229d0de484cba6fc698fc072af5/users/5c5cee4d197841c0ab1486f0afd2205a/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:11.441093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de53e4d-602a-481c-8ddf-6eb6f977b84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbeb3229d0de484cba6fc698fc072af5, user_id=5c5cee4d197841c0ab1486f0afd2205a, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:11.443792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de53e4d-602a-481c-8ddf-6eb6f977b84b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:11.459186 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1187/2356] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:47:11 2022] PUT /identity/v3/projects/cbeb3229d0de484cba6fc698fc072af5/users/5c5cee4d197841c0ab1486f0afd2205a/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:47:11.464359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0d1cd-33f7-4ba4-8682-609ac95876ff None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:11.464696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0d1cd-33f7-4ba4-8682-609ac95876ff None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:11.464960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0d1cd-33f7-4ba4-8682-609ac95876ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:11.484042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e2f0d1cd-33f7-4ba4-8682-609ac95876ff None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:11.485950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e2f0d1cd-33f7-4ba4-8682-609ac95876ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:11.549578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1170/2357] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:47:11.559285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de53e4d-602a-481c-8ddf-6eb6f977b84b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:11.561400 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7de53e4d-602a-481c-8ddf-6eb6f977b84b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:11.620356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de53e4d-602a-481c-8ddf-6eb6f977b84b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:11.624863 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915c982-1c5b-4676-ba2d-8cd156b3f857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:11.625117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915c982-1c5b-4676-ba2d-8cd156b3f857 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:11.625397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915c982-1c5b-4676-ba2d-8cd156b3f857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:11.625666 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915c982-1c5b-4676-ba2d-8cd156b3f857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:11.635597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915c982-1c5b-4676-ba2d-8cd156b3f857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:11.637114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915c982-1c5b-4676-ba2d-8cd156b3f857 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:11.646882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1188/2358] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:47:16.157005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dacc3-4a23-4d22-b7a1-a72f72542a0f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:16.157711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dacc3-4a23-4d22-b7a1-a72f72542a0f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:16.157711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dacc3-4a23-4d22-b7a1-a72f72542a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:16.208173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4d1dacc3-4a23-4d22-b7a1-a72f72542a0f None None] MFA Rules not processed for user `5c5cee4d197841c0ab1486f0afd2205a`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:16.210242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4d1dacc3-4a23-4d22-b7a1-a72f72542a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:16.297578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1171/2359] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:47:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:47:16.298734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915c982-1c5b-4676-ba2d-8cd156b3f857 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:16.312183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915c982-1c5b-4676-ba2d-8cd156b3f857 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:16.333304 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2a7bd-4a7c-4f29-85a9-ca8d49276e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:16.345994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2a7bd-4a7c-4f29-85a9-ca8d49276e38 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:16.346601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2a7bd-4a7c-4f29-85a9-ca8d49276e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:16.347059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2a7bd-4a7c-4f29-85a9-ca8d49276e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:16.348849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2a7bd-4a7c-4f29-85a9-ca8d49276e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:16.355115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2a7bd-4a7c-4f29-85a9-ca8d49276e38 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:16.395904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1189/2360] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:47:16 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:47:16.401824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dacc3-4a23-4d22-b7a1-a72f72542a0f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:16.414128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dacc3-4a23-4d22-b7a1-a72f72542a0f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:16.477201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6936db4-da8b-41b3-9f49-0528ceb1afa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:16.477971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6936db4-da8b-41b3-9f49-0528ceb1afa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:16.478256 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6936db4-da8b-41b3-9f49-0528ceb1afa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:16.478528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6936db4-da8b-41b3-9f49-0528ceb1afa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:16.479232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6936db4-da8b-41b3-9f49-0528ceb1afa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:16.481290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6936db4-da8b-41b3-9f49-0528ceb1afa4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:16.502305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1172/2361] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:47:16 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:47:16.506393 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2a7bd-4a7c-4f29-85a9-ca8d49276e38 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:16.516247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2a7bd-4a7c-4f29-85a9-ca8d49276e38 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:16.520998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2cad7-b4e5-4a36-898e-7740053999ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:16.521471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2cad7-b4e5-4a36-898e-7740053999ab admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:16.521746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2cad7-b4e5-4a36-898e-7740053999ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:16.522024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2cad7-b4e5-4a36-898e-7740053999ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:16.522751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2cad7-b4e5-4a36-898e-7740053999ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:16.524341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2cad7-b4e5-4a36-898e-7740053999ab admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:16.534859 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1190/2362] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:47:16.539203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6936db4-da8b-41b3-9f49-0528ceb1afa4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:16.550208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6936db4-da8b-41b3-9f49-0528ceb1afa4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:16.556057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851c08d-3255-4d0e-af7e-3294853a267e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:16.557062 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c08d-3255-4d0e-af7e-3294853a267e admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:16.557352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c08d-3255-4d0e-af7e-3294853a267e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:16.557479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c08d-3255-4d0e-af7e-3294853a267e admin admin] PATH_INFO: `/v3/projects/56cb31438dbf448d9a3862c45b94912e/users/5b221bf2a7a64babba0f996d3d32f321/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:16.577483 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851c08d-3255-4d0e-af7e-3294853a267e admin admin] RBAC: Authorizing `identity:create_grant(project_id=56cb31438dbf448d9a3862c45b94912e, user_id=5b221bf2a7a64babba0f996d3d32f321, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:16.580111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851c08d-3255-4d0e-af7e-3294853a267e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:16.594899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1173/2363] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:47:16 2022] PUT /identity/v3/projects/56cb31438dbf448d9a3862c45b94912e/users/5b221bf2a7a64babba0f996d3d32f321/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:47:16.599079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2cad7-b4e5-4a36-898e-7740053999ab admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:16.609047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2cad7-b4e5-4a36-898e-7740053999ab admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:16.670039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee163091-7dfc-4ba0-98fa-60f0a1c22e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:16.670895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee163091-7dfc-4ba0-98fa-60f0a1c22e9a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:16.671265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee163091-7dfc-4ba0-98fa-60f0a1c22e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:16.671630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee163091-7dfc-4ba0-98fa-60f0a1c22e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:16.672449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee163091-7dfc-4ba0-98fa-60f0a1c22e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:16.674200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee163091-7dfc-4ba0-98fa-60f0a1c22e9a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:16.684949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1191/2364] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:47:16.688977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851c08d-3255-4d0e-af7e-3294853a267e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:16.700375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851c08d-3255-4d0e-af7e-3294853a267e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:16.706427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12b274-14df-4c4d-ba8f-e6e69d6444f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:16.707339 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12b274-14df-4c4d-ba8f-e6e69d6444f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:16.707619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12b274-14df-4c4d-ba8f-e6e69d6444f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:16.707888 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12b274-14df-4c4d-ba8f-e6e69d6444f9 admin admin] PATH_INFO: `/v3/projects/56cb31438dbf448d9a3862c45b94912e/users/5b221bf2a7a64babba0f996d3d32f321/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:16.710337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12b274-14df-4c4d-ba8f-e6e69d6444f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56cb31438dbf448d9a3862c45b94912e, user_id=5b221bf2a7a64babba0f996d3d32f321, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:16.713109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12b274-14df-4c4d-ba8f-e6e69d6444f9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:16.724226 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1174/2365] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:47:16 2022] PUT /identity/v3/projects/56cb31438dbf448d9a3862c45b94912e/users/5b221bf2a7a64babba0f996d3d32f321/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:47:23.191420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db942b6d-ccd7-4a8d-920b-6e5af9085209 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:23.192047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db942b6d-ccd7-4a8d-920b-6e5af9085209 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:23.197127 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db942b6d-ccd7-4a8d-920b-6e5af9085209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:23.266504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-db942b6d-ccd7-4a8d-920b-6e5af9085209 None None] MFA Rules not processed for user `5b221bf2a7a64babba0f996d3d32f321`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:23.267955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-db942b6d-ccd7-4a8d-920b-6e5af9085209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:23.476695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1192/2366] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:47:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:47:23.482026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12b274-14df-4c4d-ba8f-e6e69d6444f9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:23.483790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9c12b274-14df-4c4d-ba8f-e6e69d6444f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:23.631935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12b274-14df-4c4d-ba8f-e6e69d6444f9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:23.638369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab13b31-6125-4c83-935b-3006e17f6e62 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6552c41f1c654ef5bfdd940e3b807dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:23.640964 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab13b31-6125-4c83-935b-3006e17f6e62 service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:23.641253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab13b31-6125-4c83-935b-3006e17f6e62 service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:23.641532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab13b31-6125-4c83-935b-3006e17f6e62 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:23.764648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab13b31-6125-4c83-935b-3006e17f6e62 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:23.766702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab13b31-6125-4c83-935b-3006e17f6e62 service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:23.789258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1175/2367] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:47:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:47:24.021747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae879f2-dafe-49ec-a29b-723e670614af None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:24.022090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae879f2-dafe-49ec-a29b-723e670614af None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:24.022295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae879f2-dafe-49ec-a29b-723e670614af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:24.053896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4ae879f2-dafe-49ec-a29b-723e670614af None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:24.056724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4ae879f2-dafe-49ec-a29b-723e670614af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:24.127849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1193/2368] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:47:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:47:24.141936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab13b31-6125-4c83-935b-3006e17f6e62 service glance] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:24.143372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3ab13b31-6125-4c83-935b-3006e17f6e62 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:24.241086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab13b31-6125-4c83-935b-3006e17f6e62 service glance] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:24.246730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1446c-840a-4ae6-acc0-93e64ad0cf15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:24.247851 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1446c-840a-4ae6-acc0-93e64ad0cf15 service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:24.248018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1446c-840a-4ae6-acc0-93e64ad0cf15 service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:24.248297 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1446c-840a-4ae6-acc0-93e64ad0cf15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:24.260527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1446c-840a-4ae6-acc0-93e64ad0cf15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:24.262164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1446c-840a-4ae6-acc0-93e64ad0cf15 service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:24.274981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1176/2369] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:47:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:47:33.064449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae879f2-dafe-49ec-a29b-723e670614af None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:33.123076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae879f2-dafe-49ec-a29b-723e670614af None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:33.126741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc06ce-b557-4663-a706-0c7ed49fca22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:33.127411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc06ce-b557-4663-a706-0c7ed49fca22 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:33.127688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc06ce-b557-4663-a706-0c7ed49fca22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:33.127958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc06ce-b557-4663-a706-0c7ed49fca22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:33.129836 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-83cc06ce-b557-4663-a706-0c7ed49fca22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:33.198592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc06ce-b557-4663-a706-0c7ed49fca22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:33.199991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc06ce-b557-4663-a706-0c7ed49fca22 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:33.210647 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1194/2370] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:47:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:47:34.172745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17404f8b-9716-4fd4-805d-ce4df507d3d7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:34.173343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17404f8b-9716-4fd4-805d-ce4df507d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:34.173343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17404f8b-9716-4fd4-805d-ce4df507d3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:34.190822 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-17404f8b-9716-4fd4-805d-ce4df507d3d7 None None] MFA Rules not processed for user `5c5cee4d197841c0ab1486f0afd2205a`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:34.192205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-17404f8b-9716-4fd4-805d-ce4df507d3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:34.199829 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1177/2371] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:47:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:47:44.997293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ee58d-40da-4043-973e-4e6aea4ba0f9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:44.997831 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ee58d-40da-4043-973e-4e6aea4ba0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:44.997915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ee58d-40da-4043-973e-4e6aea4ba0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:45.017341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ab4ee58d-40da-4043-973e-4e6aea4ba0f9 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:47:45.017876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ab4ee58d-40da-4043-973e-4e6aea4ba0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:47:45.025464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1195/2372] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:47:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:47:45.038270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17404f8b-9716-4fd4-805d-ce4df507d3d7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:47:45.050459 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17404f8b-9716-4fd4-805d-ce4df507d3d7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:47:45.055352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744ff94-dc4b-4bdb-84f8-7abdfd80eb55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:47:45.056037 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744ff94-dc4b-4bdb-84f8-7abdfd80eb55 service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:47:45.056315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744ff94-dc4b-4bdb-84f8-7abdfd80eb55 service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:47:45.057547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744ff94-dc4b-4bdb-84f8-7abdfd80eb55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:47:45.066218 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744ff94-dc4b-4bdb-84f8-7abdfd80eb55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:47:45.068554 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744ff94-dc4b-4bdb-84f8-7abdfd80eb55 service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:47:45.085517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1178/2373] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:47:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:48:17.360922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ee58d-40da-4043-973e-4e6aea4ba0f9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:17.370729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ee58d-40da-4043-973e-4e6aea4ba0f9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:17.374318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62847387-87c2-400b-874a-d1e748fcef28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:17.374965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62847387-87c2-400b-874a-d1e748fcef28 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:17.375247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62847387-87c2-400b-874a-d1e748fcef28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:17.375550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62847387-87c2-400b-874a-d1e748fcef28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:17.377365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-62847387-87c2-400b-874a-d1e748fcef28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:17.439662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62847387-87c2-400b-874a-d1e748fcef28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:17.441145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62847387-87c2-400b-874a-d1e748fcef28 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:17.451212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1196/2374] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:21.523975 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744ff94-dc4b-4bdb-84f8-7abdfd80eb55 service swift] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:21.526330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5744ff94-dc4b-4bdb-84f8-7abdfd80eb55 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:21.546597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744ff94-dc4b-4bdb-84f8-7abdfd80eb55 service swift] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:21.553417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c78cc-66db-4f31-b7dd-773d480a1c61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:21.554081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c78cc-66db-4f31-b7dd-773d480a1c61 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:21.554369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c78cc-66db-4f31-b7dd-773d480a1c61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:21.554630 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c78cc-66db-4f31-b7dd-773d480a1c61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:21.556755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-541c78cc-66db-4f31-b7dd-773d480a1c61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:21.631970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c78cc-66db-4f31-b7dd-773d480a1c61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:21.633481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c78cc-66db-4f31-b7dd-773d480a1c61 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:21.644029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1179/2375] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:21.842893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62847387-87c2-400b-874a-d1e748fcef28 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:21.852247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62847387-87c2-400b-874a-d1e748fcef28 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:21.856963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f59ab9-b68f-4bd7-a548-71e61670a989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:21.857612 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f59ab9-b68f-4bd7-a548-71e61670a989 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:21.857900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f59ab9-b68f-4bd7-a548-71e61670a989 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:21.858166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f59ab9-b68f-4bd7-a548-71e61670a989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:21.860318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-29f59ab9-b68f-4bd7-a548-71e61670a989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:21.875020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f59ab9-b68f-4bd7-a548-71e61670a989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:21.876395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f59ab9-b68f-4bd7-a548-71e61670a989 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:21.888006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1197/2376] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:31.593559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23592f2c-0f0f-4552-a4d3-9d14d24309e7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:31.594101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23592f2c-0f0f-4552-a4d3-9d14d24309e7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:31.594101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23592f2c-0f0f-4552-a4d3-9d14d24309e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:31.612784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-23592f2c-0f0f-4552-a4d3-9d14d24309e7 None None] MFA Rules not processed for user `06e1c95f942c4ce0b8487b62b3e293fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:48:31.613992 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-23592f2c-0f0f-4552-a4d3-9d14d24309e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:31.678779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1180/2377] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:48:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:48:31.921701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f59ab9-b68f-4bd7-a548-71e61670a989 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:31.930668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f59ab9-b68f-4bd7-a548-71e61670a989 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:31.935348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edbd61-0129-485d-ab20-037b299fc571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:31.935985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edbd61-0129-485d-ab20-037b299fc571 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:31.936278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edbd61-0129-485d-ab20-037b299fc571 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:31.936576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edbd61-0129-485d-ab20-037b299fc571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:31.938655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-61edbd61-0129-485d-ab20-037b299fc571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:31.949710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61edbd61-0129-485d-ab20-037b299fc571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:31.951478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61edbd61-0129-485d-ab20-037b299fc571 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:31.959443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1198/2378] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:32.528770 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23592f2c-0f0f-4552-a4d3-9d14d24309e7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:32.530510 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-23592f2c-0f0f-4552-a4d3-9d14d24309e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:32.604334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23592f2c-0f0f-4552-a4d3-9d14d24309e7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:32.609637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85168fe0-7344-4e8f-a727-0a1268666067 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e934c8e7dd0425cac68eb91e5e2c0ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:32.610297 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85168fe0-7344-4e8f-a727-0a1268666067 service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:32.610575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85168fe0-7344-4e8f-a727-0a1268666067 service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:32.610845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85168fe0-7344-4e8f-a727-0a1268666067 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:32.613860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-85168fe0-7344-4e8f-a727-0a1268666067 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:32.632720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85168fe0-7344-4e8f-a727-0a1268666067 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:32.632720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85168fe0-7344-4e8f-a727-0a1268666067 service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:32.638657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1181/2379] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:33.198996 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edbd61-0129-485d-ab20-037b299fc571 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:33.200522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-61edbd61-0129-485d-ab20-037b299fc571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:33.256624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edbd61-0129-485d-ab20-037b299fc571 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:33.260726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e187f15-598a-479a-9d82-c670a0fd0505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:33.261370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e187f15-598a-479a-9d82-c670a0fd0505 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:33.261650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e187f15-598a-479a-9d82-c670a0fd0505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:33.261921 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e187f15-598a-479a-9d82-c670a0fd0505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:33.263895 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7e187f15-598a-479a-9d82-c670a0fd0505 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:33.332409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e187f15-598a-479a-9d82-c670a0fd0505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:33.334074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e187f15-598a-479a-9d82-c670a0fd0505 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:33.344140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1199/2380] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:34.872220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e6bd-d1cb-40f0-b288-6ee0109a3cf2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:34.872785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e6bd-d1cb-40f0-b288-6ee0109a3cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:34.873023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e6bd-d1cb-40f0-b288-6ee0109a3cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:34.921445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a658e6bd-d1cb-40f0-b288-6ee0109a3cf2 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:48:34.922806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a658e6bd-d1cb-40f0-b288-6ee0109a3cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:34.936485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1182/2381] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:48:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:48:34.962866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e187f15-598a-479a-9d82-c670a0fd0505 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:34.975225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e187f15-598a-479a-9d82-c670a0fd0505 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:34.982003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da225ba4-2ced-42e3-b98e-c6499cb19bf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:34.982806 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da225ba4-2ced-42e3-b98e-c6499cb19bf5 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:34.983177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da225ba4-2ced-42e3-b98e-c6499cb19bf5 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:34.983508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da225ba4-2ced-42e3-b98e-c6499cb19bf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:34.996288 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da225ba4-2ced-42e3-b98e-c6499cb19bf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:35.001506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da225ba4-2ced-42e3-b98e-c6499cb19bf5 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:35.012655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1200/2382] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:48:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Apr 29 18:48:39.041115 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658e6bd-d1cb-40f0-b288-6ee0109a3cf2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:39.052843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658e6bd-d1cb-40f0-b288-6ee0109a3cf2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:39.063387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d6278-a5e0-45aa-9680-4231eccffa46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:39.064222 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6278-a5e0-45aa-9680-4231eccffa46 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:39.064710 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6278-a5e0-45aa-9680-4231eccffa46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:39.065205 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6278-a5e0-45aa-9680-4231eccffa46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:39.067503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-df0d6278-a5e0-45aa-9680-4231eccffa46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:39.147400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d6278-a5e0-45aa-9680-4231eccffa46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:39.149210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d6278-a5e0-45aa-9680-4231eccffa46 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:39.158898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1183/2383] 10.222.0.50 () {60 vars in 1396 bytes} [Fri Apr 29 18:48:39 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:49.861561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da225ba4-2ced-42e3-b98e-c6499cb19bf5 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:49.875522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da225ba4-2ced-42e3-b98e-c6499cb19bf5 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:49.879507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103e361-6716-4438-8df8-cabc10c4570a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:49.880337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103e361-6716-4438-8df8-cabc10c4570a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:49.880665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103e361-6716-4438-8df8-cabc10c4570a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:49.881108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103e361-6716-4438-8df8-cabc10c4570a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:49.883807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3103e361-6716-4438-8df8-cabc10c4570a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:49.893822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103e361-6716-4438-8df8-cabc10c4570a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:49.895236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103e361-6716-4438-8df8-cabc10c4570a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:49.903948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1201/2384] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:57.729165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de492031-0ad5-4249-bb1c-b3db8d06c720 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:57.729767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de492031-0ad5-4249-bb1c-b3db8d06c720 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:57.729767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de492031-0ad5-4249-bb1c-b3db8d06c720 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:57.731065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1184/2385] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:48:57.737367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1351c30-9182-422f-a64e-4d4aba1cd4e3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:57.737846 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1351c30-9182-422f-a64e-4d4aba1cd4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:57.737964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1351c30-9182-422f-a64e-4d4aba1cd4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:57.776006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f1351c30-9182-422f-a64e-4d4aba1cd4e3 None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:48:57.777259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f1351c30-9182-422f-a64e-4d4aba1cd4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:57.787275 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1202/2386] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:48:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:48:57.799854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de492031-0ad5-4249-bb1c-b3db8d06c720 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:57.811523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de492031-0ad5-4249-bb1c-b3db8d06c720 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:57.815242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d52d11-ba93-45f8-aa2f-3f61b295733c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6552c41f1c654ef5bfdd940e3b807dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:57.815889 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d52d11-ba93-45f8-aa2f-3f61b295733c service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:57.816166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d52d11-ba93-45f8-aa2f-3f61b295733c service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:57.816438 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d52d11-ba93-45f8-aa2f-3f61b295733c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:57.827948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d52d11-ba93-45f8-aa2f-3f61b295733c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:57.829624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d52d11-ba93-45f8-aa2f-3f61b295733c service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:57.841770 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1185/2387] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:57.921114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d0f77-7320-4a77-bca1-f504adaf6c91 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:57.921405 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d0f77-7320-4a77-bca1-f504adaf6c91 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:57.921683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d0f77-7320-4a77-bca1-f504adaf6c91 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:57.923081 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1203/2388] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:48:57.928036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc420bb9-87be-488c-b782-9c6d7c667c3e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:57.928339 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc420bb9-87be-488c-b782-9c6d7c667c3e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:57.928638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc420bb9-87be-488c-b782-9c6d7c667c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:57.948422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bc420bb9-87be-488c-b782-9c6d7c667c3e None None] MFA Rules not processed for user `07849b2925334763b119111ed52dd34c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:48:57.949651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bc420bb9-87be-488c-b782-9c6d7c667c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:57.956680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1186/2389] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:48:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:48:57.968770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d0f77-7320-4a77-bca1-f504adaf6c91 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:57.981265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d0f77-7320-4a77-bca1-f504adaf6c91 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:57.985333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fb95e-b7c9-4c7c-b35c-92981b21f407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:57.985969 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fb95e-b7c9-4c7c-b35c-92981b21f407 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:57.986249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fb95e-b7c9-4c7c-b35c-92981b21f407 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:57.986523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fb95e-b7c9-4c7c-b35c-92981b21f407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:57.993585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fb95e-b7c9-4c7c-b35c-92981b21f407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:57.995009 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fb95e-b7c9-4c7c-b35c-92981b21f407 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:58.002644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1204/2390] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:58.098521 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60c0b2-1c6a-4c58-96e7-9891bdd0f884 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:58.098808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60c0b2-1c6a-4c58-96e7-9891bdd0f884 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:58.099078 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60c0b2-1c6a-4c58-96e7-9891bdd0f884 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:58.100453 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1187/2391] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:48:58.106052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac03e7-4016-4244-bf9a-eac511fa324d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:58.106336 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac03e7-4016-4244-bf9a-eac511fa324d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:58.106607 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac03e7-4016-4244-bf9a-eac511fa324d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:58.126504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-37ac03e7-4016-4244-bf9a-eac511fa324d None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:48:58.127711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-37ac03e7-4016-4244-bf9a-eac511fa324d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:48:58.139056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1205/2392] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:48:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:48:58.152425 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60c0b2-1c6a-4c58-96e7-9891bdd0f884 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:58.165700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60c0b2-1c6a-4c58-96e7-9891bdd0f884 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:58.170108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21065e4f-05f6-48cc-8d89-f6e47ce46976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:58.170750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21065e4f-05f6-48cc-8d89-f6e47ce46976 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:58.171023 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21065e4f-05f6-48cc-8d89-f6e47ce46976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:58.171294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21065e4f-05f6-48cc-8d89-f6e47ce46976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:58.181883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21065e4f-05f6-48cc-8d89-f6e47ce46976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:58.183311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21065e4f-05f6-48cc-8d89-f6e47ce46976 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:58.194547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1188/2393] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:48:59.863988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac03e7-4016-4244-bf9a-eac511fa324d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:48:59.873971 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac03e7-4016-4244-bf9a-eac511fa324d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:48:59.878235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafde43-ec8f-41fc-addf-0b992e2dc6c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:48:59.878916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafde43-ec8f-41fc-addf-0b992e2dc6c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:48:59.879198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafde43-ec8f-41fc-addf-0b992e2dc6c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:48:59.879537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafde43-ec8f-41fc-addf-0b992e2dc6c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:48:59.889322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafde43-ec8f-41fc-addf-0b992e2dc6c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:48:59.890745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafde43-ec8f-41fc-addf-0b992e2dc6c0 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:48:59.900253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1206/2394] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:48:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:49:03.662406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21065e4f-05f6-48cc-8d89-f6e47ce46976 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:49:03.665231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-21065e4f-05f6-48cc-8d89-f6e47ce46976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:49:03.691668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21065e4f-05f6-48cc-8d89-f6e47ce46976 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:49:03.709137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40bd83-315a-400b-b872-69ac2b699d17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:49:03.710473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40bd83-315a-400b-b872-69ac2b699d17 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:49:03.710963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40bd83-315a-400b-b872-69ac2b699d17 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:49:03.711441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40bd83-315a-400b-b872-69ac2b699d17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:49:03.713745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cf40bd83-315a-400b-b872-69ac2b699d17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:49:03.728626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40bd83-315a-400b-b872-69ac2b699d17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:49:03.731568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40bd83-315a-400b-b872-69ac2b699d17 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:49:03.746643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1189/2395] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:49:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:49:15.599450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96761e-30cc-4e25-8152-e14ad723d7e3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:49:15.600008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96761e-30cc-4e25-8152-e14ad723d7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:49:15.600008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96761e-30cc-4e25-8152-e14ad723d7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:49:15.625828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6c96761e-30cc-4e25-8152-e14ad723d7e3 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:49:15.627919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6c96761e-30cc-4e25-8152-e14ad723d7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:49:15.640244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1207/2396] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:49:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:49:15.652860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40bd83-315a-400b-b872-69ac2b699d17 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:49:15.664660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40bd83-315a-400b-b872-69ac2b699d17 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:49:15.671351 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e791f-23c2-4c7d-9e8f-a9339aaa8b42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:49:15.671996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e791f-23c2-4c7d-9e8f-a9339aaa8b42 service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:49:15.672272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e791f-23c2-4c7d-9e8f-a9339aaa8b42 service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:49:15.672553 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e791f-23c2-4c7d-9e8f-a9339aaa8b42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:49:15.681925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e791f-23c2-4c7d-9e8f-a9339aaa8b42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:49:15.683583 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e791f-23c2-4c7d-9e8f-a9339aaa8b42 service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:49:15.691686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1190/2397] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:49:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:49:56.986946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96761e-30cc-4e25-8152-e14ad723d7e3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:49:56.988614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6c96761e-30cc-4e25-8152-e14ad723d7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:49:57.005989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96761e-30cc-4e25-8152-e14ad723d7e3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:49:57.011233 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a244c-a72a-4685-b24f-e6038d415076 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:49:57.011922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a244c-a72a-4685-b24f-e6038d415076 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:49:57.012197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a244c-a72a-4685-b24f-e6038d415076 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:49:57.012470 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a244c-a72a-4685-b24f-e6038d415076 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:49:57.014345 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2f5a244c-a72a-4685-b24f-e6038d415076 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:49:57.026879 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a244c-a72a-4685-b24f-e6038d415076 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:49:57.028550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a244c-a72a-4685-b24f-e6038d415076 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:49:57.039231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1208/2398] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:49:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:50:04.204756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b283e-56fc-4280-82bc-dbf54b5b7736 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:50:04.205321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b283e-56fc-4280-82bc-dbf54b5b7736 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:50:04.205321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b283e-56fc-4280-82bc-dbf54b5b7736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:50:04.224153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c13b283e-56fc-4280-82bc-dbf54b5b7736 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:50:04.225366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c13b283e-56fc-4280-82bc-dbf54b5b7736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:50:04.234214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1191/2399] 10.222.0.50 () {58 vars in 967 bytes} [Fri Apr 29 18:50:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:50:04.257388 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a244c-a72a-4685-b24f-e6038d415076 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:50:04.269103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a244c-a72a-4685-b24f-e6038d415076 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:50:04.274851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da71e60-4f9f-4096-a3ea-fe7f9b768b10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:50:04.275543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da71e60-4f9f-4096-a3ea-fe7f9b768b10 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:50:04.276063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da71e60-4f9f-4096-a3ea-fe7f9b768b10 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:50:04.276063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da71e60-4f9f-4096-a3ea-fe7f9b768b10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:50:04.287896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da71e60-4f9f-4096-a3ea-fe7f9b768b10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:50:04.289832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da71e60-4f9f-4096-a3ea-fe7f9b768b10 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:50:04.299157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1209/2400] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:50:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:50:23.119236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b283e-56fc-4280-82bc-dbf54b5b7736 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:50:23.121466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c13b283e-56fc-4280-82bc-dbf54b5b7736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:50:23.186814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b283e-56fc-4280-82bc-dbf54b5b7736 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:50:23.191793 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:50:23.192700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:50:23.192966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:50:23.193240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] PATH_INFO: `/v3/users/06e1c95f942c4ce0b8487b62b3e293fb` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:50:23.194827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e1c95f942c4ce0b8487b62b3e293fb)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:50:23.196787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:50:23.245299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e1c95f942c4ce0b8487b62b3e293fb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:50:23.245949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e1c95f942c4ce0b8487b62b3e293fb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:50:23.257102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e1c95f942c4ce0b8487b62b3e293fb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:50:23.263769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e1c95f942c4ce0b8487b62b3e293fb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:50:23.272213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e1c95f942c4ce0b8487b62b3e293fb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:50:23.278725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1192/2401] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:50:23 2022] DELETE /identity/v3/users/06e1c95f942c4ce0b8487b62b3e293fb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:50:23.458472 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da71e60-4f9f-4096-a3ea-fe7f9b768b10 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:50:23.470866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da71e60-4f9f-4096-a3ea-fe7f9b768b10 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:50:23.531305 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:50:23.532203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:50:23.532510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:50:23.532787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] PATH_INFO: `/v3/projects/941e38a2e73a4905ac8568f1af5af294` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:50:23.533982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=941e38a2e73a4905ac8568f1af5af294)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:50:23.535987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:50:23.593655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '941e38a2e73a4905ac8568f1af5af294', 'request_id': 'req-8991f71a-82d0-4344-8313-dabf861176ed'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:50:23.603013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '941e38a2e73a4905ac8568f1af5af294', 'request_id': 'req-8991f71a-82d0-4344-8313-dabf861176ed'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:50:23.609767 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '941e38a2e73a4905ac8568f1af5af294', 'request_id': 'req-8991f71a-82d0-4344-8313-dabf861176ed'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:50:23.619789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] The token cache is being invalidate because project 941e38a2e73a4905ac8568f1af5af294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:50:23.620071 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:50:23.620757 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:50:23.622327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1210/2402] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:50:23 2022] DELETE /identity/v3/projects/941e38a2e73a4905ac8568f1af5af294 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:50:43.005616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:50:43.008203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:50:43.093775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cc37b-63a3-49e3-9024-5b5794585540 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:50:43.166409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca41e32-fa45-43f0-ae9c-81f17abaebea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:50:43.167074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca41e32-fa45-43f0-ae9c-81f17abaebea service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:50:43.167356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca41e32-fa45-43f0-ae9c-81f17abaebea service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:50:43.167626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca41e32-fa45-43f0-ae9c-81f17abaebea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:50:43.169431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5ca41e32-fa45-43f0-ae9c-81f17abaebea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:50:43.227066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca41e32-fa45-43f0-ae9c-81f17abaebea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:50:43.228797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca41e32-fa45-43f0-ae9c-81f17abaebea service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:50:43.238260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1193/2403] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:50:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:51:14.886912 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:51:14.889443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:14.910703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991f71a-82d0-4344-8313-dabf861176ed admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:51:14.918119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882fc0b-18ef-488e-8f12-e5296b2039be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:51:14.918771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882fc0b-18ef-488e-8f12-e5296b2039be service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:14.919052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882fc0b-18ef-488e-8f12-e5296b2039be service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:14.919322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882fc0b-18ef-488e-8f12-e5296b2039be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:14.921605 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6882fc0b-18ef-488e-8f12-e5296b2039be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:14.983917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882fc0b-18ef-488e-8f12-e5296b2039be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:51:14.985542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882fc0b-18ef-488e-8f12-e5296b2039be service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:51:14.995112 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1211/2404] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:51:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:51:15.098804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f38cc-f05d-4b3a-b234-c91056bca777 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:15.099329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f38cc-f05d-4b3a-b234-c91056bca777 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:15.099429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f38cc-f05d-4b3a-b234-c91056bca777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:15.120447 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dc7f38cc-f05d-4b3a-b234-c91056bca777 None None] MFA Rules not processed for user `e9a2443e71c54075823f75fdb8df469e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:51:15.121936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dc7f38cc-f05d-4b3a-b234-c91056bca777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:15.129632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1194/2405] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:51:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:51:35.560535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882fc0b-18ef-488e-8f12-e5296b2039be service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:51:35.562681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6882fc0b-18ef-488e-8f12-e5296b2039be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:35.631228 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882fc0b-18ef-488e-8f12-e5296b2039be service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:51:35.635969 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a2d24-b90f-4bda-be65-40bdb5e9ea8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:51:35.636748 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a2d24-b90f-4bda-be65-40bdb5e9ea8a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:35.637098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a2d24-b90f-4bda-be65-40bdb5e9ea8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:35.637346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a2d24-b90f-4bda-be65-40bdb5e9ea8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:35.639470 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-305a2d24-b90f-4bda-be65-40bdb5e9ea8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:35.652652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a2d24-b90f-4bda-be65-40bdb5e9ea8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:51:35.654135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a2d24-b90f-4bda-be65-40bdb5e9ea8a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:51:35.663427 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1212/2406] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:51:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:51:35.783618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc5928-b78e-45eb-be16-c1420e9a033d None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:35.784260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc5928-b78e-45eb-be16-c1420e9a033d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:35.784260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc5928-b78e-45eb-be16-c1420e9a033d None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:35.785693 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1195/2407] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:51:35.791339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe21ad-e75c-4456-8cea-ccad85c44253 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:35.791625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe21ad-e75c-4456-8cea-ccad85c44253 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:35.791893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe21ad-e75c-4456-8cea-ccad85c44253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:35.809857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6bbe21ad-e75c-4456-8cea-ccad85c44253 None None] MFA Rules not processed for user `07849b2925334763b119111ed52dd34c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:51:35.811174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6bbe21ad-e75c-4456-8cea-ccad85c44253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:35.818576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1213/2408] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:51:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:51:35.832485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc5928-b78e-45eb-be16-c1420e9a033d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:51:35.834284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-10fc5928-b78e-45eb-be16-c1420e9a033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:35.848561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc5928-b78e-45eb-be16-c1420e9a033d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:51:35.852366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9c155-9ee8-470e-b118-75939b59e21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:51:35.853086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9c155-9ee8-470e-b118-75939b59e21c service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:35.853370 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9c155-9ee8-470e-b118-75939b59e21c service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:35.853639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9c155-9ee8-470e-b118-75939b59e21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:35.861106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9c155-9ee8-470e-b118-75939b59e21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:51:35.862522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9c155-9ee8-470e-b118-75939b59e21c service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:51:35.870283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1196/2409] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:51:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:51:35.970021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48677865-6ab7-499e-a6f5-da9fd68ce811 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:35.970275 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48677865-6ab7-499e-a6f5-da9fd68ce811 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:35.970565 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48677865-6ab7-499e-a6f5-da9fd68ce811 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:35.971976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1214/2410] 10.222.0.50 () {54 vars in 921 bytes} [Fri Apr 29 18:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Apr 29 18:51:35.977458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad1ed7-6d3e-4186-884f-0503be1c67e9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:35.977745 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad1ed7-6d3e-4186-884f-0503be1c67e9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:35.978010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad1ed7-6d3e-4186-884f-0503be1c67e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:35.997333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ddad1ed7-6d3e-4186-884f-0503be1c67e9 None None] MFA Rules not processed for user `341358254d884b278d7fa19d95c2dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:51:35.998520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ddad1ed7-6d3e-4186-884f-0503be1c67e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:36.009170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1197/2411] 10.222.0.50 () {58 vars in 1018 bytes} [Fri Apr 29 18:51:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:51:36.019805 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48677865-6ab7-499e-a6f5-da9fd68ce811 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:51:36.032203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48677865-6ab7-499e-a6f5-da9fd68ce811 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:51:36.036776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd6bdb-2756-4dae-ad13-ec9d8e280c2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:51:36.037414 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd6bdb-2756-4dae-ad13-ec9d8e280c2c service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:36.037700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd6bdb-2756-4dae-ad13-ec9d8e280c2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:36.037964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd6bdb-2756-4dae-ad13-ec9d8e280c2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:36.048633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd6bdb-2756-4dae-ad13-ec9d8e280c2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:51:36.050086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd6bdb-2756-4dae-ad13-ec9d8e280c2c service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:51:36.060640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1215/2412] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:51:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:51:45.408788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad1ed7-6d3e-4186-884f-0503be1c67e9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:51:45.410794 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ddad1ed7-6d3e-4186-884f-0503be1c67e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:45.485397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad1ed7-6d3e-4186-884f-0503be1c67e9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:51:45.491178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390d30b-967a-48ff-a387-1b9e053a3e05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e934c8e7dd0425cac68eb91e5e2c0ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:51:45.491853 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390d30b-967a-48ff-a387-1b9e053a3e05 service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:45.492133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390d30b-967a-48ff-a387-1b9e053a3e05 service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:45.492402 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390d30b-967a-48ff-a387-1b9e053a3e05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:45.494323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9390d30b-967a-48ff-a387-1b9e053a3e05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:45.508157 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390d30b-967a-48ff-a387-1b9e053a3e05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:51:45.510103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390d30b-967a-48ff-a387-1b9e053a3e05 service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:51:45.522532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1198/2413] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:51:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:51:49.708131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aa7ac-9a54-4c94-9c61-9dafe7dd141b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:49.708696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aa7ac-9a54-4c94-9c61-9dafe7dd141b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:49.708696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aa7ac-9a54-4c94-9c61-9dafe7dd141b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:49.730036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1d2aa7ac-9a54-4c94-9c61-9dafe7dd141b None None] MFA Rules not processed for user `49aaea67fba64d2583d2023505ce7779`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:51:49.731234 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1d2aa7ac-9a54-4c94-9c61-9dafe7dd141b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:49.786631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1216/2414] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:51:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:51:58.162942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390d30b-967a-48ff-a387-1b9e053a3e05 service placement] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:51:58.165503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9390d30b-967a-48ff-a387-1b9e053a3e05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:58.183971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390d30b-967a-48ff-a387-1b9e053a3e05 service placement] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:51:58.189987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604d583-dfae-41ef-b599-d04e2bb8d624 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:51:58.190624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604d583-dfae-41ef-b599-d04e2bb8d624 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:51:58.190900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604d583-dfae-41ef-b599-d04e2bb8d624 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:51:58.191164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604d583-dfae-41ef-b599-d04e2bb8d624 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:51:58.193381 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2604d583-dfae-41ef-b599-d04e2bb8d624 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:51:58.251399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604d583-dfae-41ef-b599-d04e2bb8d624 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:51:58.253626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604d583-dfae-41ef-b599-d04e2bb8d624 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:51:58.267915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1199/2415] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:51:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:52:07.449944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aa7ac-9a54-4c94-9c61-9dafe7dd141b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:52:07.463347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aa7ac-9a54-4c94-9c61-9dafe7dd141b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:52:07.469440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f60e54-26f6-436d-91b2-9a29dcd7b5de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:52:07.470088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f60e54-26f6-436d-91b2-9a29dcd7b5de service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:52:07.470480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f60e54-26f6-436d-91b2-9a29dcd7b5de service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:52:07.470749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f60e54-26f6-436d-91b2-9a29dcd7b5de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:52:07.472914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-93f60e54-26f6-436d-91b2-9a29dcd7b5de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:52:07.485390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f60e54-26f6-436d-91b2-9a29dcd7b5de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:52:07.486966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f60e54-26f6-436d-91b2-9a29dcd7b5de service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:52:07.497281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1217/2416] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:52:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:52:07.802150 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217875c7-e63d-4aac-8180-4bcb87471f05 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:52:07.802721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217875c7-e63d-4aac-8180-4bcb87471f05 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:52:07.802721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217875c7-e63d-4aac-8180-4bcb87471f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:52:07.821232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-217875c7-e63d-4aac-8180-4bcb87471f05 None None] MFA Rules not processed for user `a1569db96d824c0e9f16b386df5e3413`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:52:07.822672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-217875c7-e63d-4aac-8180-4bcb87471f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:52:07.829872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1200/2417] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:52:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:52:56.253841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f60e54-26f6-436d-91b2-9a29dcd7b5de service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:52:56.264996 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f60e54-26f6-436d-91b2-9a29dcd7b5de service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:52:56.271313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e2021-d110-481d-9723-87b5f12b8c0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:52:56.271967 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e2021-d110-481d-9723-87b5f12b8c0a service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:52:56.272249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e2021-d110-481d-9723-87b5f12b8c0a service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:52:56.272550 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e2021-d110-481d-9723-87b5f12b8c0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:52:56.274379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-050e2021-d110-481d-9723-87b5f12b8c0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:52:56.335994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e2021-d110-481d-9723-87b5f12b8c0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:52:56.337636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e2021-d110-481d-9723-87b5f12b8c0a service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:52:56.347441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1218/2418] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:52:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:52:57.582315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d1b98-641d-4ef3-abe2-87f204ea272c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:52:57.583026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d1b98-641d-4ef3-abe2-87f204ea272c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:52:57.583026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d1b98-641d-4ef3-abe2-87f204ea272c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:52:57.607778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5b7d1b98-641d-4ef3-abe2-87f204ea272c None None] MFA Rules not processed for user `64c8eec4efab42bfa5d9e9b157883dd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:52:57.609364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5b7d1b98-641d-4ef3-abe2-87f204ea272c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:52:57.682284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1201/2419] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:52:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:53:11.856920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889073ef-1e2c-48a1-adf4-0781ddc0dce8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:11.857409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889073ef-1e2c-48a1-adf4-0781ddc0dce8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:11.857503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889073ef-1e2c-48a1-adf4-0781ddc0dce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:11.875014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-889073ef-1e2c-48a1-adf4-0781ddc0dce8 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:53:11.876214 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-889073ef-1e2c-48a1-adf4-0781ddc0dce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:11.933409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1219/2420] 10.222.0.50 () {58 vars in 967 bytes} [Fri Apr 29 18:53:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:53:11.993364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d1b98-641d-4ef3-abe2-87f204ea272c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:11.994994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5b7d1b98-641d-4ef3-abe2-87f204ea272c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:12.065271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d1b98-641d-4ef3-abe2-87f204ea272c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:12.072026 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f80f14-e33b-4182-b8c1-863e31e2ce92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:12.072712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f80f14-e33b-4182-b8c1-863e31e2ce92 service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:12.072995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f80f14-e33b-4182-b8c1-863e31e2ce92 service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:12.073272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f80f14-e33b-4182-b8c1-863e31e2ce92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:12.081750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f80f14-e33b-4182-b8c1-863e31e2ce92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:12.083343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f80f14-e33b-4182-b8c1-863e31e2ce92 service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:12.091430 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1202/2421] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:53:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:53:12.503871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889073ef-1e2c-48a1-adf4-0781ddc0dce8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:12.513277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889073ef-1e2c-48a1-adf4-0781ddc0dce8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:12.516851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7413a-aae1-4dc4-bb2e-94b59c467d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:12.517496 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7413a-aae1-4dc4-bb2e-94b59c467d64 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:12.517776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7413a-aae1-4dc4-bb2e-94b59c467d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:12.518049 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7413a-aae1-4dc4-bb2e-94b59c467d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:12.519959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4cf7413a-aae1-4dc4-bb2e-94b59c467d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:12.583440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7413a-aae1-4dc4-bb2e-94b59c467d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:12.584915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7413a-aae1-4dc4-bb2e-94b59c467d64 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:12.594774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1220/2422] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:53:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:53:14.895581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f80f14-e33b-4182-b8c1-863e31e2ce92 service swift] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:14.905727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f80f14-e33b-4182-b8c1-863e31e2ce92 service swift] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:14.910044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e4594-8f81-4ae1-a914-8a99a02cf473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:14.910879 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e4594-8f81-4ae1-a914-8a99a02cf473 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:14.911532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e4594-8f81-4ae1-a914-8a99a02cf473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:14.911836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e4594-8f81-4ae1-a914-8a99a02cf473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:14.914798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f82e4594-8f81-4ae1-a914-8a99a02cf473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:14.934516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e4594-8f81-4ae1-a914-8a99a02cf473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:14.936304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e4594-8f81-4ae1-a914-8a99a02cf473 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:14.952552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1203/2423] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:53:20.335527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7413a-aae1-4dc4-bb2e-94b59c467d64 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:20.339956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4cf7413a-aae1-4dc4-bb2e-94b59c467d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:20.353629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7413a-aae1-4dc4-bb2e-94b59c467d64 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:20.370798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:20.372518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:20.373000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:20.373702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] PATH_INFO: `/v3/users/e9a2443e71c54075823f75fdb8df469e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:20.392924 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a2443e71c54075823f75fdb8df469e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:20.396085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:20.464673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a2443e71c54075823f75fdb8df469e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:20.465725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a2443e71c54075823f75fdb8df469e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:20.481405 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a2443e71c54075823f75fdb8df469e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:20.494587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a2443e71c54075823f75fdb8df469e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:20.504561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ee95b020-04d4-4bf1-827f-e7c46184074b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a2443e71c54075823f75fdb8df469e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:20.513920 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1221/2424] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:53:20 2022] DELETE /identity/v3/users/e9a2443e71c54075823f75fdb8df469e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:53:20.818530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e4594-8f81-4ae1-a914-8a99a02cf473 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:20.833545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e4594-8f81-4ae1-a914-8a99a02cf473 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:20.926178 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:20.928220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:20.928997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:20.929467 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] PATH_INFO: `/v3/projects/e2f2dec75acb4277945a986b8393b0a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:20.946368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f2dec75acb4277945a986b8393b0a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:20.949967 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.023825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f2dec75acb4277945a986b8393b0a1', 'request_id': 'req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.043484 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f2dec75acb4277945a986b8393b0a1', 'request_id': 'req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.051935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f2dec75acb4277945a986b8393b0a1', 'request_id': 'req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.063768 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] The token cache is being invalidate because project e2f2dec75acb4277945a986b8393b0a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:53:21.064524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.065569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-51c2dcdc-c41c-40c4-b489-84cdbfd025d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.067655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1204/2425] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:53:20 2022] DELETE /identity/v3/projects/e2f2dec75acb4277945a986b8393b0a1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:53:21.073463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26045a50-6da3-43d8-b231-69ef4ec95a65 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.073747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26045a50-6da3-43d8-b231-69ef4ec95a65 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.074242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26045a50-6da3-43d8-b231-69ef4ec95a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.095953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-26045a50-6da3-43d8-b231-69ef4ec95a65 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:53:21.097567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26045a50-6da3-43d8-b231-69ef4ec95a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:21.163752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1222/2426] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:53:21.171813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debd637-c3c1-4330-947c-9b7120b32b25 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.171813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debd637-c3c1-4330-947c-9b7120b32b25 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.172034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debd637-c3c1-4330-947c-9b7120b32b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.194202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5debd637-c3c1-4330-947c-9b7120b32b25 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:53:21.195670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5debd637-c3c1-4330-947c-9b7120b32b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:21.199435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26045a50-6da3-43d8-b231-69ef4ec95a65 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:21.201279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26045a50-6da3-43d8-b231-69ef4ec95a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:21.219588 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1205/2427] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:53:21.220045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26045a50-6da3-43d8-b231-69ef4ec95a65 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:21.223485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debd637-c3c1-4330-947c-9b7120b32b25 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:21.230529 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:21.232323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.232653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.233077 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] PATH_INFO: `/v3/users/5c5cee4d197841c0ab1486f0afd2205a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.234854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5cee4d197841c0ab1486f0afd2205a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:21.236999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debd637-c3c1-4330-947c-9b7120b32b25 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:21.237162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.242061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ca54a-15f2-4cf1-8e2d-adc421fca711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:21.243084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ca54a-15f2-4cf1-8e2d-adc421fca711 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.243590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ca54a-15f2-4cf1-8e2d-adc421fca711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.244075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ca54a-15f2-4cf1-8e2d-adc421fca711 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.245068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ca54a-15f2-4cf1-8e2d-adc421fca711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:53:21.245578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ca54a-15f2-4cf1-8e2d-adc421fca711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:21.247442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ca54a-15f2-4cf1-8e2d-adc421fca711 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.260793 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1206/2428] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:53:21.264192 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ca54a-15f2-4cf1-8e2d-adc421fca711 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:21.279942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ca54a-15f2-4cf1-8e2d-adc421fca711 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:21.287981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c34b7-8c06-4813-8605-ebfca7dd07aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:21.289739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c34b7-8c06-4813-8605-ebfca7dd07aa admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.290277 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c34b7-8c06-4813-8605-ebfca7dd07aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.290813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c34b7-8c06-4813-8605-ebfca7dd07aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.292432 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c34b7-8c06-4813-8605-ebfca7dd07aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:21.295029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c34b7-8c06-4813-8605-ebfca7dd07aa admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.301613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5cee4d197841c0ab1486f0afd2205a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.302366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5cee4d197841c0ab1486f0afd2205a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.319401 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5cee4d197841c0ab1486f0afd2205a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.333874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5cee4d197841c0ab1486f0afd2205a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.344761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1207/2429] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:53:21 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:53:21.345932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5cee4d197841c0ab1486f0afd2205a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.349233 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c34b7-8c06-4813-8605-ebfca7dd07aa admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:21.352832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1223/2430] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:53:21 2022] DELETE /identity/v3/users/5c5cee4d197841c0ab1486f0afd2205a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:53:21.353684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d20c34b7-8c06-4813-8605-ebfca7dd07aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:21.433535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c34b7-8c06-4813-8605-ebfca7dd07aa admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:21.499948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f6a52-0e2c-42b4-9937-f25a53e7f62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:21.501363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f6a52-0e2c-42b4-9937-f25a53e7f62c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.501826 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f6a52-0e2c-42b4-9937-f25a53e7f62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.502098 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f6a52-0e2c-42b4-9937-f25a53e7f62c admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.502798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f6a52-0e2c-42b4-9937-f25a53e7f62c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:21.504936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f6a52-0e2c-42b4-9937-f25a53e7f62c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.527996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1208/2431] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:53:21 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:53:21.532378 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:21.544651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ab781-6adc-426d-b989-9da7297c0b38 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:21.549578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d464e-a61e-44cf-8ac0-48303c30e302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:21.550324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d464e-a61e-44cf-8ac0-48303c30e302 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.550616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d464e-a61e-44cf-8ac0-48303c30e302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.550871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d464e-a61e-44cf-8ac0-48303c30e302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.551846 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d464e-a61e-44cf-8ac0-48303c30e302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:21.553600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d464e-a61e-44cf-8ac0-48303c30e302 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.566910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1224/2432] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:53:21.571817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f6a52-0e2c-42b4-9937-f25a53e7f62c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:21.577073 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d464e-a61e-44cf-8ac0-48303c30e302 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:21.579547 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7c4d464e-a61e-44cf-8ac0-48303c30e302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:21.585702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f6a52-0e2c-42b4-9937-f25a53e7f62c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:21.591289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17316df7-ba90-4f69-b4c6-014a5626645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:21.592071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316df7-ba90-4f69-b4c6-014a5626645e admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.598639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316df7-ba90-4f69-b4c6-014a5626645e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.598996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316df7-ba90-4f69-b4c6-014a5626645e admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.600835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d464e-a61e-44cf-8ac0-48303c30e302 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:21.606983 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:21.608139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.608420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.608724 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17316df7-ba90-4f69-b4c6-014a5626645e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:21.611728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17316df7-ba90-4f69-b4c6-014a5626645e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.616447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] PATH_INFO: `/v3/projects/cbeb3229d0de484cba6fc698fc072af5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.618913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbeb3229d0de484cba6fc698fc072af5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:21.621217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.629401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1209/2433] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:53:21.636309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17316df7-ba90-4f69-b4c6-014a5626645e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:21.651497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17316df7-ba90-4f69-b4c6-014a5626645e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:21.657374 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760aa9b-03a5-4af3-a9c9-21477be3543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:21.658445 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760aa9b-03a5-4af3-a9c9-21477be3543d admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.658665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760aa9b-03a5-4af3-a9c9-21477be3543d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.658896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760aa9b-03a5-4af3-a9c9-21477be3543d admin admin] PATH_INFO: `/v3/projects/9da6a94754bf43c4a0119272fb920c06/users/cc76dcb94459489f965c9205f2d461cb/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.676373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760aa9b-03a5-4af3-a9c9-21477be3543d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9da6a94754bf43c4a0119272fb920c06, user_id=cc76dcb94459489f965c9205f2d461cb, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:21.678966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760aa9b-03a5-4af3-a9c9-21477be3543d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.680531 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbeb3229d0de484cba6fc698fc072af5', 'request_id': 'req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.690357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbeb3229d0de484cba6fc698fc072af5', 'request_id': 'req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.696275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1210/2434] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:53:21 2022] PUT /identity/v3/projects/9da6a94754bf43c4a0119272fb920c06/users/cc76dcb94459489f965c9205f2d461cb/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:53:21.699095 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbeb3229d0de484cba6fc698fc072af5', 'request_id': 'req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.701334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c145172-f8f8-4f6b-a0f9-8c4e70bd637e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.701657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c145172-f8f8-4f6b-a0f9-8c4e70bd637e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.701906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c145172-f8f8-4f6b-a0f9-8c4e70bd637e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.710816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] The token cache is being invalidate because project cbeb3229d0de484cba6fc698fc072af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:53:21.711256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.711989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.714287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1225/2435] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:53:21 2022] DELETE /identity/v3/projects/cbeb3229d0de484cba6fc698fc072af5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:53:21.718468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:21.720053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:21.724566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3c145172-f8f8-4f6b-a0f9-8c4e70bd637e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:53:21.726632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3c145172-f8f8-4f6b-a0f9-8c4e70bd637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:21.794553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcabb8-77c6-4a7f-a606-81cbac9ea82f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:21.796609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1211/2436] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:53:21.799414 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:21.800298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.800605 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.800878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] PATH_INFO: `/v3/users/5b221bf2a7a64babba0f996d3d32f321` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.802523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b221bf2a7a64babba0f996d3d32f321)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:21.804897 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c145172-f8f8-4f6b-a0f9-8c4e70bd637e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:21.805141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.810464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3c145172-f8f8-4f6b-a0f9-8c4e70bd637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:21.865003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b221bf2a7a64babba0f996d3d32f321'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.865689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b221bf2a7a64babba0f996d3d32f321'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.871536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c145172-f8f8-4f6b-a0f9-8c4e70bd637e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:21.877578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c52c8-438e-46d6-aa62-a1c2e4b81957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:21.878245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c52c8-438e-46d6-aa62-a1c2e4b81957 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:21.878523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c52c8-438e-46d6-aa62-a1c2e4b81957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:21.878793 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c52c8-438e-46d6-aa62-a1c2e4b81957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:21.880105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b221bf2a7a64babba0f996d3d32f321'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.880772 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c52c52c8-438e-46d6-aa62-a1c2e4b81957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:21.888906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b221bf2a7a64babba0f996d3d32f321'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.894123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c52c8-438e-46d6-aa62-a1c2e4b81957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:21.895734 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c52c8-438e-46d6-aa62-a1c2e4b81957 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:21.897389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b221bf2a7a64babba0f996d3d32f321'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:21.904132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1226/2437] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:53:21 2022] DELETE /identity/v3/users/5b221bf2a7a64babba0f996d3d32f321 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:53:21.909466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1212/2438] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:53:22.114268 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:22.115928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:22.201937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a5e83-e485-4225-9e4b-40b28b97aa9a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:22.206467 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:22.207339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:22.207615 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:22.207883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] PATH_INFO: `/v3/projects/56cb31438dbf448d9a3862c45b94912e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:22.209087 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56cb31438dbf448d9a3862c45b94912e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:22.211022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:22.260769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56cb31438dbf448d9a3862c45b94912e', 'request_id': 'req-a906461a-1171-4030-8aaf-17baa4eab3a2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:22.268873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56cb31438dbf448d9a3862c45b94912e', 'request_id': 'req-a906461a-1171-4030-8aaf-17baa4eab3a2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:22.275369 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56cb31438dbf448d9a3862c45b94912e', 'request_id': 'req-a906461a-1171-4030-8aaf-17baa4eab3a2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:22.286297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] The token cache is being invalidate because project 56cb31438dbf448d9a3862c45b94912e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:53:22.286586 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:22.287319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:53:22.289055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1227/2439] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:53:22 2022] DELETE /identity/v3/projects/56cb31438dbf448d9a3862c45b94912e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:53:25.844301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880b17c-8a97-44aa-b24c-add25f97eeaa None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:25.844877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880b17c-8a97-44aa-b24c-add25f97eeaa None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:25.845087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880b17c-8a97-44aa-b24c-add25f97eeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:25.876897 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6880b17c-8a97-44aa-b24c-add25f97eeaa None None] MFA Rules not processed for user `cc76dcb94459489f965c9205f2d461cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:53:25.878258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6880b17c-8a97-44aa-b24c-add25f97eeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:25.946899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1213/2440] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:53:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:53:25.958771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:25.960168 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:26.033847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906461a-1171-4030-8aaf-17baa4eab3a2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:26.039564 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042038f6-e33c-4307-9dee-77f6c411dbe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:26.040236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042038f6-e33c-4307-9dee-77f6c411dbe6 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:26.040678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042038f6-e33c-4307-9dee-77f6c411dbe6 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:26.041115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042038f6-e33c-4307-9dee-77f6c411dbe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:26.053088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042038f6-e33c-4307-9dee-77f6c411dbe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:26.054857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042038f6-e33c-4307-9dee-77f6c411dbe6 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:26.065133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1228/2441] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:53:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:53:27.684909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5967746-5a59-4a59-b630-9a5c6e3f38ab None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:27.685446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5967746-5a59-4a59-b630-9a5c6e3f38ab None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:27.685585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5967746-5a59-4a59-b630-9a5c6e3f38ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:27.704074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f5967746-5a59-4a59-b630-9a5c6e3f38ab None None] MFA Rules not processed for user `cc76dcb94459489f965c9205f2d461cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:53:27.705461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f5967746-5a59-4a59-b630-9a5c6e3f38ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:27.713898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1214/2442] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:53:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:53:27.755368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042038f6-e33c-4307-9dee-77f6c411dbe6 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:27.757250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-042038f6-e33c-4307-9dee-77f6c411dbe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:27.819459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042038f6-e33c-4307-9dee-77f6c411dbe6 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:27.823935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414c683-6cf3-4066-afd2-a04bd5270484 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6552c41f1c654ef5bfdd940e3b807dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:27.824533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414c683-6cf3-4066-afd2-a04bd5270484 service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:27.824829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414c683-6cf3-4066-afd2-a04bd5270484 service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:27.825097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414c683-6cf3-4066-afd2-a04bd5270484 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:27.826969 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c414c683-6cf3-4066-afd2-a04bd5270484 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:27.841939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414c683-6cf3-4066-afd2-a04bd5270484 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:27.843398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414c683-6cf3-4066-afd2-a04bd5270484 service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:27.855866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1229/2443] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:53:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:53:28.527293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5967746-5a59-4a59-b630-9a5c6e3f38ab None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:28.531953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f5967746-5a59-4a59-b630-9a5c6e3f38ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:28.630784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5967746-5a59-4a59-b630-9a5c6e3f38ab None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:28.637018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431d908-48cf-4dae-91a8-66cc28fe7b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:28.638190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431d908-48cf-4dae-91a8-66cc28fe7b28 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:28.638803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431d908-48cf-4dae-91a8-66cc28fe7b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:28.639215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431d908-48cf-4dae-91a8-66cc28fe7b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:28.641242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8431d908-48cf-4dae-91a8-66cc28fe7b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:28.659802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431d908-48cf-4dae-91a8-66cc28fe7b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:28.661489 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431d908-48cf-4dae-91a8-66cc28fe7b28 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:28.675515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1215/2444] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:53:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:53:52.988833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414c683-6cf3-4066-afd2-a04bd5270484 service glance] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:53.000455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414c683-6cf3-4066-afd2-a04bd5270484 service glance] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:53.007286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a678c-4853-43a6-b321-1b139bcfdebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:53.007932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a678c-4853-43a6-b321-1b139bcfdebc service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:53.008212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a678c-4853-43a6-b321-1b139bcfdebc service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:53.008481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a678c-4853-43a6-b321-1b139bcfdebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:53.010384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-db7a678c-4853-43a6-b321-1b139bcfdebc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:53.092218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a678c-4853-43a6-b321-1b139bcfdebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:53.093854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a678c-4853-43a6-b321-1b139bcfdebc service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:53.113525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1230/2445] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:53:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:53:54.045705 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431d908-48cf-4dae-91a8-66cc28fe7b28 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:53:54.055305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431d908-48cf-4dae-91a8-66cc28fe7b28 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:53:54.060113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20645f6f-ad48-48e7-a2f6-34849f85051a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:53:54.061039 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20645f6f-ad48-48e7-a2f6-34849f85051a service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:53:54.061613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20645f6f-ad48-48e7-a2f6-34849f85051a service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:53:54.062043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20645f6f-ad48-48e7-a2f6-34849f85051a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:53:54.065079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-20645f6f-ad48-48e7-a2f6-34849f85051a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:53:54.086947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20645f6f-ad48-48e7-a2f6-34849f85051a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:53:54.088644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20645f6f-ad48-48e7-a2f6-34849f85051a service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:53:54.098050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1216/2446] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:53:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:54:00.093476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a678c-4853-43a6-b321-1b139bcfdebc service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:00.095111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-db7a678c-4853-43a6-b321-1b139bcfdebc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:00.107527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a678c-4853-43a6-b321-1b139bcfdebc service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:00.111837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954458f-e429-41df-b099-d4f7e40a0a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:00.112581 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954458f-e429-41df-b099-d4f7e40a0a4b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:00.112866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954458f-e429-41df-b099-d4f7e40a0a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:00.113140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954458f-e429-41df-b099-d4f7e40a0a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:00.115044 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f954458f-e429-41df-b099-d4f7e40a0a4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:00.176059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954458f-e429-41df-b099-d4f7e40a0a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:00.177511 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954458f-e429-41df-b099-d4f7e40a0a4b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:00.187136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1231/2447] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:54:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:54:06.619494 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20645f6f-ad48-48e7-a2f6-34849f85051a service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:06.621112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-20645f6f-ad48-48e7-a2f6-34849f85051a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:06.636956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20645f6f-ad48-48e7-a2f6-34849f85051a service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:06.641931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:06.642755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:06.643037 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:06.643312 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] PATH_INFO: `/v3/users/0ad000d2e49f4b6fa0e9315d35e7d4da` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:06.644550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad000d2e49f4b6fa0e9315d35e7d4da)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:06.646491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:06.697171 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad000d2e49f4b6fa0e9315d35e7d4da'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:06.697897 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad000d2e49f4b6fa0e9315d35e7d4da'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:06.710324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad000d2e49f4b6fa0e9315d35e7d4da'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:06.717435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad000d2e49f4b6fa0e9315d35e7d4da'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:06.727060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad000d2e49f4b6fa0e9315d35e7d4da'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:06.733269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1217/2448] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:54:06 2022] DELETE /identity/v3/users/0ad000d2e49f4b6fa0e9315d35e7d4da => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:54:06.912299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954458f-e429-41df-b099-d4f7e40a0a4b service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:06.922672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954458f-e429-41df-b099-d4f7e40a0a4b service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:06.979732 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:06.980649 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:06.980901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:06.981208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] PATH_INFO: `/v3/projects/131b4943a4aa42a8b09d614d80f552cc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:06.982475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] RBAC: Authorizing `identity:delete_project(project_id=131b4943a4aa42a8b09d614d80f552cc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:06.984642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:07.030439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '131b4943a4aa42a8b09d614d80f552cc', 'request_id': 'req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.039979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '131b4943a4aa42a8b09d614d80f552cc', 'request_id': 'req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.048198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '131b4943a4aa42a8b09d614d80f552cc', 'request_id': 'req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.058795 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] The token cache is being invalidate because project 131b4943a4aa42a8b09d614d80f552cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:54:07.059079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.059730 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.061422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1232/2449] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:54:06 2022] DELETE /identity/v3/projects/131b4943a4aa42a8b09d614d80f552cc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:54:07.066249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:07.067698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:07.081167 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e5548-91d1-4e60-a1b7-3a68bff7642f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:07.137361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:07.138268 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:07.138550 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:07.138849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] PATH_INFO: `/v3/users/8324178d3bbc4ddeb5d5873d2893a84b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:07.140166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8324178d3bbc4ddeb5d5873d2893a84b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:07.142128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:07.189255 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8324178d3bbc4ddeb5d5873d2893a84b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.189937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8324178d3bbc4ddeb5d5873d2893a84b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.202542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8324178d3bbc4ddeb5d5873d2893a84b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.209017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8324178d3bbc4ddeb5d5873d2893a84b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.217309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c445ecb9-cec3-4e83-b521-d86fe22067b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8324178d3bbc4ddeb5d5873d2893a84b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.223099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1218/2450] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:54:07 2022] DELETE /identity/v3/users/8324178d3bbc4ddeb5d5873d2893a84b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:54:07.401638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:07.403080 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:07.416203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff2bb5-80c3-42cc-a744-1f3c5800545b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:07.475525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:07.476395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:07.476698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:07.476967 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] PATH_INFO: `/v3/projects/17cf3832459944abacf72169a8abd09a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:07.478170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] RBAC: Authorizing `identity:delete_project(project_id=17cf3832459944abacf72169a8abd09a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:07.480061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:07.524548 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17cf3832459944abacf72169a8abd09a', 'request_id': 'req-087e499c-898d-4920-b75c-68efaa37f43e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.532408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17cf3832459944abacf72169a8abd09a', 'request_id': 'req-087e499c-898d-4920-b75c-68efaa37f43e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.538827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17cf3832459944abacf72169a8abd09a', 'request_id': 'req-087e499c-898d-4920-b75c-68efaa37f43e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.549847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] The token cache is being invalidate because project 17cf3832459944abacf72169a8abd09a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:54:07.550133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.550894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:54:07.552609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1233/2451] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:54:07 2022] DELETE /identity/v3/projects/17cf3832459944abacf72169a8abd09a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:54:22.557428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45483aba-691f-41cf-915a-d61eff2a7359 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:22.558011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45483aba-691f-41cf-915a-d61eff2a7359 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:22.558011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45483aba-691f-41cf-915a-d61eff2a7359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:22.579767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-45483aba-691f-41cf-915a-d61eff2a7359 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:54:22.581087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-45483aba-691f-41cf-915a-d61eff2a7359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:22.640397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1219/2452] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:54:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:54:22.691449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:22.692889 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:22.764503 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e499c-898d-4920-b75c-68efaa37f43e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:22.769756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee89a83-e831-4669-85ad-932d66564097 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:22.770410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee89a83-e831-4669-85ad-932d66564097 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:22.770850 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee89a83-e831-4669-85ad-932d66564097 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:22.770953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee89a83-e831-4669-85ad-932d66564097 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:22.780905 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee89a83-e831-4669-85ad-932d66564097 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:22.783906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee89a83-e831-4669-85ad-932d66564097 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:22.792476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1234/2453] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:54:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:54:40.477805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45483aba-691f-41cf-915a-d61eff2a7359 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:40.479258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-45483aba-691f-41cf-915a-d61eff2a7359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:40.540824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45483aba-691f-41cf-915a-d61eff2a7359 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:40.544787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c5121-3cb9-4553-8e56-62d428696632 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6552c41f1c654ef5bfdd940e3b807dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:40.545989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c5121-3cb9-4553-8e56-62d428696632 service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:40.546274 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c5121-3cb9-4553-8e56-62d428696632 service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:40.546545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c5121-3cb9-4553-8e56-62d428696632 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:40.548470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-916c5121-3cb9-4553-8e56-62d428696632 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:40.630786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c5121-3cb9-4553-8e56-62d428696632 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:40.632241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c5121-3cb9-4553-8e56-62d428696632 service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:40.642817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1220/2454] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:54:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:54:40.708440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee89a83-e831-4669-85ad-932d66564097 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:40.709913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4ee89a83-e831-4669-85ad-932d66564097 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:40.766417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee89a83-e831-4669-85ad-932d66564097 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:40.770331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873fcfd-f7d1-4736-baa2-a80c9f8acde8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:40.770962 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873fcfd-f7d1-4736-baa2-a80c9f8acde8 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:40.771212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873fcfd-f7d1-4736-baa2-a80c9f8acde8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:40.771515 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873fcfd-f7d1-4736-baa2-a80c9f8acde8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:40.773553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c873fcfd-f7d1-4736-baa2-a80c9f8acde8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:40.783848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873fcfd-f7d1-4736-baa2-a80c9f8acde8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:40.785298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873fcfd-f7d1-4736-baa2-a80c9f8acde8 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:40.792969 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1235/2455] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:54:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:54:40.870757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c5121-3cb9-4553-8e56-62d428696632 service glance] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:40.872291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-916c5121-3cb9-4553-8e56-62d428696632 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:40.885394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c5121-3cb9-4553-8e56-62d428696632 service glance] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:40.890141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d735d-8abe-4862-bcdd-912a07ea24ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:40.890798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d735d-8abe-4862-bcdd-912a07ea24ee service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:40.891074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d735d-8abe-4862-bcdd-912a07ea24ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:40.891324 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d735d-8abe-4862-bcdd-912a07ea24ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:40.893492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-aa1d735d-8abe-4862-bcdd-912a07ea24ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:40.907618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d735d-8abe-4862-bcdd-912a07ea24ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:40.909104 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d735d-8abe-4862-bcdd-912a07ea24ee service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:40.920022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1221/2456] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:54:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:54:40.985396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873fcfd-f7d1-4736-baa2-a80c9f8acde8 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:40.987995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c873fcfd-f7d1-4736-baa2-a80c9f8acde8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:41.065892 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873fcfd-f7d1-4736-baa2-a80c9f8acde8 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:41.071872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3676a0-537c-478d-8f71-e1b356cd3d0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e934c8e7dd0425cac68eb91e5e2c0ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:41.072510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3676a0-537c-478d-8f71-e1b356cd3d0e service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:41.072815 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3676a0-537c-478d-8f71-e1b356cd3d0e service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:41.073087 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3676a0-537c-478d-8f71-e1b356cd3d0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:41.075106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5e3676a0-537c-478d-8f71-e1b356cd3d0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:41.086661 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3676a0-537c-478d-8f71-e1b356cd3d0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:41.088230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3676a0-537c-478d-8f71-e1b356cd3d0e service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:41.098203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1236/2457] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:54:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:54:42.854364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b8bc3-a742-4204-b659-049ed8147a53 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:42.854872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b8bc3-a742-4204-b659-049ed8147a53 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:42.854872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b8bc3-a742-4204-b659-049ed8147a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:42.872586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-074b8bc3-a742-4204-b659-049ed8147a53 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:54:42.873886 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-074b8bc3-a742-4204-b659-049ed8147a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:54:42.883263 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1222/2458] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:54:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:54:42.893104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3676a0-537c-478d-8f71-e1b356cd3d0e service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:54:42.904220 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3676a0-537c-478d-8f71-e1b356cd3d0e service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:54:42.909703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffb54f-2291-4778-b999-260a4fb5f405 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:54:42.910354 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb54f-2291-4778-b999-260a4fb5f405 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:54:42.910638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb54f-2291-4778-b999-260a4fb5f405 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:54:42.910910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb54f-2291-4778-b999-260a4fb5f405 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:54:42.919294 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffb54f-2291-4778-b999-260a4fb5f405 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:54:42.921000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffb54f-2291-4778-b999-260a4fb5f405 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:54:42.929168 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1237/2459] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:54:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:55:03.064422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b8bc3-a742-4204-b659-049ed8147a53 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:03.066725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-074b8bc3-a742-4204-b659-049ed8147a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:03.088716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b8bc3-a742-4204-b659-049ed8147a53 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:03.095591 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8ec2a-d113-43f4-bc99-7f6efdcc0a2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:03.096241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8ec2a-d113-43f4-bc99-7f6efdcc0a2a service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:03.096517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8ec2a-d113-43f4-bc99-7f6efdcc0a2a service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:03.096887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8ec2a-d113-43f4-bc99-7f6efdcc0a2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:03.098808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3bd8ec2a-d113-43f4-bc99-7f6efdcc0a2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:03.191857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8ec2a-d113-43f4-bc99-7f6efdcc0a2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:03.193407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8ec2a-d113-43f4-bc99-7f6efdcc0a2a service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:03.211796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1223/2460] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:55:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:55:07.095671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffb54f-2291-4778-b999-260a4fb5f405 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:07.109300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffb54f-2291-4778-b999-260a4fb5f405 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:07.115237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f2274-90d5-4657-8631-6c841a0a7dfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:07.116093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f2274-90d5-4657-8631-6c841a0a7dfd service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:07.116373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f2274-90d5-4657-8631-6c841a0a7dfd service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:07.116682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f2274-90d5-4657-8631-6c841a0a7dfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:07.118432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4e0f2274-90d5-4657-8631-6c841a0a7dfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:07.130604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f2274-90d5-4657-8631-6c841a0a7dfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:07.132177 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f2274-90d5-4657-8631-6c841a0a7dfd service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:07.144167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1238/2461] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:55:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:55:07.243465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acf699-1c7c-42e8-bf9f-7516b08b70c4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:07.244034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acf699-1c7c-42e8-bf9f-7516b08b70c4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:07.244134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acf699-1c7c-42e8-bf9f-7516b08b70c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:07.262256 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-32acf699-1c7c-42e8-bf9f-7516b08b70c4 None None] MFA Rules not processed for user `a1569db96d824c0e9f16b386df5e3413`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:07.263603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-32acf699-1c7c-42e8-bf9f-7516b08b70c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:07.325780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1224/2462] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:55:07.703944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f2274-90d5-4657-8631-6c841a0a7dfd service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:07.713133 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f2274-90d5-4657-8631-6c841a0a7dfd service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:07.719238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b948cd-3fd6-4251-ad49-96ac322e443f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:07.719903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b948cd-3fd6-4251-ad49-96ac322e443f service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:07.720184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b948cd-3fd6-4251-ad49-96ac322e443f service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:07.720453 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b948cd-3fd6-4251-ad49-96ac322e443f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:07.722479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-02b948cd-3fd6-4251-ad49-96ac322e443f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:07.734530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b948cd-3fd6-4251-ad49-96ac322e443f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:07.735974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b948cd-3fd6-4251-ad49-96ac322e443f service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:07.744179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1239/2463] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:55:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:55:09.854369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7b23d-1a42-4eed-aa7e-01fc55da6a3d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:09.854867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7b23d-1a42-4eed-aa7e-01fc55da6a3d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:09.854972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7b23d-1a42-4eed-aa7e-01fc55da6a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:09.886504 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-52e7b23d-1a42-4eed-aa7e-01fc55da6a3d None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:09.887824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-52e7b23d-1a42-4eed-aa7e-01fc55da6a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:09.899334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1225/2464] 10.222.0.50 () {58 vars in 967 bytes} [Fri Apr 29 18:55:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:55:09.913216 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b948cd-3fd6-4251-ad49-96ac322e443f service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:09.936831 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b948cd-3fd6-4251-ad49-96ac322e443f service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:09.942932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e4cdd-ec56-4c06-a069-284a75c47fab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:09.943662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e4cdd-ec56-4c06-a069-284a75c47fab service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:09.944026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e4cdd-ec56-4c06-a069-284a75c47fab service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:09.944426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e4cdd-ec56-4c06-a069-284a75c47fab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:09.953645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e4cdd-ec56-4c06-a069-284a75c47fab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:09.955239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e4cdd-ec56-4c06-a069-284a75c47fab service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:09.963645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1240/2465] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:55:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:55:12.211707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7b23d-1a42-4eed-aa7e-01fc55da6a3d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:12.219141 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-52e7b23d-1a42-4eed-aa7e-01fc55da6a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:12.245815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7b23d-1a42-4eed-aa7e-01fc55da6a3d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:12.251996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68934e2-c2f3-4cee-b837-f86c0dd7e115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:12.252787 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68934e2-c2f3-4cee-b837-f86c0dd7e115 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:12.253134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68934e2-c2f3-4cee-b837-f86c0dd7e115 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:12.253478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68934e2-c2f3-4cee-b837-f86c0dd7e115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:12.255476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f68934e2-c2f3-4cee-b837-f86c0dd7e115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:12.269165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68934e2-c2f3-4cee-b837-f86c0dd7e115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:12.274018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68934e2-c2f3-4cee-b837-f86c0dd7e115 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:12.288326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1226/2466] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:55:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:55:35.505783 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e4cdd-ec56-4c06-a069-284a75c47fab service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:35.515320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e4cdd-ec56-4c06-a069-284a75c47fab service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:35.519808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22913f29-253e-4c36-bf2b-644948cdb8f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:35.520523 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913f29-253e-4c36-bf2b-644948cdb8f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:35.520814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913f29-253e-4c36-bf2b-644948cdb8f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:35.521074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913f29-253e-4c36-bf2b-644948cdb8f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:35.523018 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-22913f29-253e-4c36-bf2b-644948cdb8f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:35.590035 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22913f29-253e-4c36-bf2b-644948cdb8f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:35.591468 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22913f29-253e-4c36-bf2b-644948cdb8f8 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:35.601406 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1241/2467] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:55:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:55:48.065437 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68934e2-c2f3-4cee-b837-f86c0dd7e115 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:48.104727 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68934e2-c2f3-4cee-b837-f86c0dd7e115 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:48.109372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f1f24-57d1-4b5a-a61b-4f2f34b6aeaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:48.110041 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f1f24-57d1-4b5a-a61b-4f2f34b6aeaa service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:48.110321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f1f24-57d1-4b5a-a61b-4f2f34b6aeaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:48.110590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f1f24-57d1-4b5a-a61b-4f2f34b6aeaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:48.112958 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c28f1f24-57d1-4b5a-a61b-4f2f34b6aeaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:48.233997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f1f24-57d1-4b5a-a61b-4f2f34b6aeaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:48.235465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f1f24-57d1-4b5a-a61b-4f2f34b6aeaa service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:48.255795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1227/2468] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:55:51.536438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22913f29-253e-4c36-bf2b-644948cdb8f8 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:51.538109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-22913f29-253e-4c36-bf2b-644948cdb8f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:51.593488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22913f29-253e-4c36-bf2b-644948cdb8f8 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:51.599430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:51.600287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:51.600611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:51.600871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] PATH_INFO: `/v3/users/64c8eec4efab42bfa5d9e9b157883dd7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:51.602117 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c8eec4efab42bfa5d9e9b157883dd7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:51.604023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:51.652789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c8eec4efab42bfa5d9e9b157883dd7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:51.653432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c8eec4efab42bfa5d9e9b157883dd7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:51.664306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c8eec4efab42bfa5d9e9b157883dd7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:51.673023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c8eec4efab42bfa5d9e9b157883dd7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:51.680084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89535742-fa41-4710-8f8f-c383f24c49c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c8eec4efab42bfa5d9e9b157883dd7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:51.685759 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1242/2469] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:55:51 2022] DELETE /identity/v3/users/64c8eec4efab42bfa5d9e9b157883dd7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:51.880177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f1f24-57d1-4b5a-a61b-4f2f34b6aeaa service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:51.893491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f1f24-57d1-4b5a-a61b-4f2f34b6aeaa service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:51.953439 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:51.954337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:51.954616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:51.954886 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] PATH_INFO: `/v3/projects/8d9c212254b240af938e5d7c2e9e1261` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:51.956310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9c212254b240af938e5d7c2e9e1261)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:51.958438 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.007087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9c212254b240af938e5d7c2e9e1261', 'request_id': 'req-06d78483-613c-4ec9-a793-5d23e15c2d59'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:52.016420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9c212254b240af938e5d7c2e9e1261', 'request_id': 'req-06d78483-613c-4ec9-a793-5d23e15c2d59'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:52.024072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9c212254b240af938e5d7c2e9e1261', 'request_id': 'req-06d78483-613c-4ec9-a793-5d23e15c2d59'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:52.034755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] The token cache is being invalidate because project 8d9c212254b240af938e5d7c2e9e1261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:55:52.035017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:52.035754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-06d78483-613c-4ec9-a793-5d23e15c2d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:52.037335 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1228/2470] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:55:51 2022] DELETE /identity/v3/projects/8d9c212254b240af938e5d7c2e9e1261 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:52.042457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08923d-8c3f-4b31-b33b-a59c1cab8228 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.042751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08923d-8c3f-4b31-b33b-a59c1cab8228 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.043023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08923d-8c3f-4b31-b33b-a59c1cab8228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.100827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-df08923d-8c3f-4b31-b33b-a59c1cab8228 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:52.102006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-df08923d-8c3f-4b31-b33b-a59c1cab8228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:52.163250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1243/2471] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:55:52.171594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed88df-9077-49a0-a476-8c32fcc64d67 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.171882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed88df-9077-49a0-a476-8c32fcc64d67 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.172151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed88df-9077-49a0-a476-8c32fcc64d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.192202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3fed88df-9077-49a0-a476-8c32fcc64d67 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:52.193441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3fed88df-9077-49a0-a476-8c32fcc64d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:52.202109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1229/2472] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:55:52.206575 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08923d-8c3f-4b31-b33b-a59c1cab8228 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.217770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08923d-8c3f-4b31-b33b-a59c1cab8228 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:52.222421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf40435-8a78-4dc5-a038-061288d4926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:52.223073 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf40435-8a78-4dc5-a038-061288d4926a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.223355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf40435-8a78-4dc5-a038-061288d4926a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.223621 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf40435-8a78-4dc5-a038-061288d4926a admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.224346 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf40435-8a78-4dc5-a038-061288d4926a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:55:52.224683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf40435-8a78-4dc5-a038-061288d4926a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:52.226307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf40435-8a78-4dc5-a038-061288d4926a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.238774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1244/2473] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:55:52.243776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed88df-9077-49a0-a476-8c32fcc64d67 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.259849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed88df-9077-49a0-a476-8c32fcc64d67 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:52.268708 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5478e-4192-4595-9259-744bd22b13d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:52.269368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5478e-4192-4595-9259-744bd22b13d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.269644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5478e-4192-4595-9259-744bd22b13d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.269909 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5478e-4192-4595-9259-744bd22b13d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.270594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc5478e-4192-4595-9259-744bd22b13d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:52.272582 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc5478e-4192-4595-9259-744bd22b13d7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.302857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1230/2474] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:55:52 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:55:52.307310 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf40435-8a78-4dc5-a038-061288d4926a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.319203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf40435-8a78-4dc5-a038-061288d4926a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:52.378707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a2a53-2356-4421-97c1-c793db5de2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:52.379521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a2a53-2356-4421-97c1-c793db5de2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.379874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a2a53-2356-4421-97c1-c793db5de2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.380213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a2a53-2356-4421-97c1-c793db5de2e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.380938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a2a53-2356-4421-97c1-c793db5de2e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:52.382880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a2a53-2356-4421-97c1-c793db5de2e3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.406149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1245/2475] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:55:52 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:55:52.411165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5478e-4192-4595-9259-744bd22b13d7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.421822 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5478e-4192-4595-9259-744bd22b13d7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:52.427072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33277bd-72fe-41ab-bc5e-78ca1df58c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:52.427712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33277bd-72fe-41ab-bc5e-78ca1df58c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.427990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33277bd-72fe-41ab-bc5e-78ca1df58c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.428265 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33277bd-72fe-41ab-bc5e-78ca1df58c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.429019 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33277bd-72fe-41ab-bc5e-78ca1df58c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:52.430636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33277bd-72fe-41ab-bc5e-78ca1df58c96 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.442264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1231/2476] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:55:52.446424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a2a53-2356-4421-97c1-c793db5de2e3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.457248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a2a53-2356-4421-97c1-c793db5de2e3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:52.462397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb942904-8b46-4336-9084-f23232084c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:52.463050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb942904-8b46-4336-9084-f23232084c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.463333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb942904-8b46-4336-9084-f23232084c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.463603 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb942904-8b46-4336-9084-f23232084c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.464314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb942904-8b46-4336-9084-f23232084c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:52.465940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb942904-8b46-4336-9084-f23232084c15 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.477042 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1246/2477] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:55:52.482232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33277bd-72fe-41ab-bc5e-78ca1df58c96 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.498094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33277bd-72fe-41ab-bc5e-78ca1df58c96 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:52.504929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeacf9-c154-48fb-af30-5a8a91736512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:52.505806 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeacf9-c154-48fb-af30-5a8a91736512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.506065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeacf9-c154-48fb-af30-5a8a91736512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.506347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeacf9-c154-48fb-af30-5a8a91736512 admin admin] PATH_INFO: `/v3/projects/7ab3e2eec8ae4b338ab5c514d4bcac74/users/7212dab8c95f42ba9baa91579fdb5cc9/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.522641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceeacf9-c154-48fb-af30-5a8a91736512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab3e2eec8ae4b338ab5c514d4bcac74, user_id=7212dab8c95f42ba9baa91579fdb5cc9, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:52.525322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceeacf9-c154-48fb-af30-5a8a91736512 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.540995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1232/2478] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:55:52 2022] PUT /identity/v3/projects/7ab3e2eec8ae4b338ab5c514d4bcac74/users/7212dab8c95f42ba9baa91579fdb5cc9/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:52.552019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8d36-47e0-4e0a-89df-45681b2e9420 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.552536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8d36-47e0-4e0a-89df-45681b2e9420 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.552676 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8d36-47e0-4e0a-89df-45681b2e9420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.586643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d54e8d36-47e0-4e0a-89df-45681b2e9420 None None] MFA Rules not processed for user `7212dab8c95f42ba9baa91579fdb5cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:52.588203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d54e8d36-47e0-4e0a-89df-45681b2e9420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:52.659750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1247/2479] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:55:52.664687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeacf9-c154-48fb-af30-5a8a91736512 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.727986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeacf9-c154-48fb-af30-5a8a91736512 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:52.733703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62801f25-12cf-469e-90d3-e0cac7147b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:52.734356 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62801f25-12cf-469e-90d3-e0cac7147b61 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.734635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62801f25-12cf-469e-90d3-e0cac7147b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.734907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62801f25-12cf-469e-90d3-e0cac7147b61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.735584 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62801f25-12cf-469e-90d3-e0cac7147b61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:52.737568 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62801f25-12cf-469e-90d3-e0cac7147b61 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.765701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1233/2480] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:55:52 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:55:52.770059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e8d36-47e0-4e0a-89df-45681b2e9420 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.780970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e8d36-47e0-4e0a-89df-45681b2e9420 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:52.838622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587e5c4-6e3f-4437-801c-b5aa7737985f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:52.839210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587e5c4-6e3f-4437-801c-b5aa7737985f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.839490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587e5c4-6e3f-4437-801c-b5aa7737985f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.839764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587e5c4-6e3f-4437-801c-b5aa7737985f admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.840450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587e5c4-6e3f-4437-801c-b5aa7737985f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:52.842421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587e5c4-6e3f-4437-801c-b5aa7737985f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.867449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1248/2481] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:55:52 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:55:52.869829 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62801f25-12cf-469e-90d3-e0cac7147b61 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.881564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62801f25-12cf-469e-90d3-e0cac7147b61 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:52.887344 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee8545-033e-434e-a2c2-daef7934324c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:52.888027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee8545-033e-434e-a2c2-daef7934324c admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.888310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee8545-033e-434e-a2c2-daef7934324c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.888621 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee8545-033e-434e-a2c2-daef7934324c admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.889616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee8545-033e-434e-a2c2-daef7934324c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:52.891283 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee8545-033e-434e-a2c2-daef7934324c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.903155 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1234/2482] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:55:52.907543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587e5c4-6e3f-4437-801c-b5aa7737985f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.919229 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587e5c4-6e3f-4437-801c-b5aa7737985f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:52.924829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925823fe-d959-4829-9423-2258f09a7471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:52.925734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925823fe-d959-4829-9423-2258f09a7471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:52.926048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925823fe-d959-4829-9423-2258f09a7471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:52.926309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925823fe-d959-4829-9423-2258f09a7471 admin admin] PATH_INFO: `/v3/projects/70c44f4979a147a7aad2d936414efb0d/users/1bd178e4ac154682a773be7107e802cf/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:52.942142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925823fe-d959-4829-9423-2258f09a7471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c44f4979a147a7aad2d936414efb0d, user_id=1bd178e4ac154682a773be7107e802cf, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:52.944808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925823fe-d959-4829-9423-2258f09a7471 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:52.959749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1249/2483] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:55:52 2022] PUT /identity/v3/projects/70c44f4979a147a7aad2d936414efb0d/users/1bd178e4ac154682a773be7107e802cf/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:52.963820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee8545-033e-434e-a2c2-daef7934324c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:52.974897 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee8545-033e-434e-a2c2-daef7934324c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:53.033270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84008e16-b574-4c0d-8d60-99bfa5eb8f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:53.034243 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84008e16-b574-4c0d-8d60-99bfa5eb8f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:53.034646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84008e16-b574-4c0d-8d60-99bfa5eb8f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:53.035017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84008e16-b574-4c0d-8d60-99bfa5eb8f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:53.036140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84008e16-b574-4c0d-8d60-99bfa5eb8f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:53.038739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84008e16-b574-4c0d-8d60-99bfa5eb8f00 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:53.052461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1235/2484] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:55:53.057234 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925823fe-d959-4829-9423-2258f09a7471 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:53.068314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925823fe-d959-4829-9423-2258f09a7471 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:53.073847 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fe91e-a060-4df7-b802-6320f88996de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:53.074737 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fe91e-a060-4df7-b802-6320f88996de admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:53.075013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fe91e-a060-4df7-b802-6320f88996de admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:53.075285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fe91e-a060-4df7-b802-6320f88996de admin admin] PATH_INFO: `/v3/projects/70c44f4979a147a7aad2d936414efb0d/users/1bd178e4ac154682a773be7107e802cf/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:53.077459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fe91e-a060-4df7-b802-6320f88996de admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c44f4979a147a7aad2d936414efb0d, user_id=1bd178e4ac154682a773be7107e802cf, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:53.080138 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fe91e-a060-4df7-b802-6320f88996de admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:53.091236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1250/2485] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:55:53 2022] PUT /identity/v3/projects/70c44f4979a147a7aad2d936414efb0d/users/1bd178e4ac154682a773be7107e802cf/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:53.101694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc0b20-5261-43be-9a14-cffa4aac2a37 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:53.101984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc0b20-5261-43be-9a14-cffa4aac2a37 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:53.102258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc0b20-5261-43be-9a14-cffa4aac2a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:53.130585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0efc0b20-5261-43be-9a14-cffa4aac2a37 None None] MFA Rules not processed for user `1bd178e4ac154682a773be7107e802cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:53.132174 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0efc0b20-5261-43be-9a14-cffa4aac2a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:53.200473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1236/2486] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:55:53.214218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fe91e-a060-4df7-b802-6320f88996de admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:53.217320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-403fe91e-a060-4df7-b802-6320f88996de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:53.278694 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fe91e-a060-4df7-b802-6320f88996de admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:53.283317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea50c1b-be5f-4aad-8924-888f4ff0e2c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:53.284029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea50c1b-be5f-4aad-8924-888f4ff0e2c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:53.284298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea50c1b-be5f-4aad-8924-888f4ff0e2c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:53.284630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea50c1b-be5f-4aad-8924-888f4ff0e2c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:53.294232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea50c1b-be5f-4aad-8924-888f4ff0e2c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:53.295647 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea50c1b-be5f-4aad-8924-888f4ff0e2c3 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:53.305260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1251/2487] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:55:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:55:53.699317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc0b20-5261-43be-9a14-cffa4aac2a37 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:53.763364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc0b20-5261-43be-9a14-cffa4aac2a37 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:53.768871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:53.770045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:53.770316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:53.770579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] PATH_INFO: `/v3/users/7212dab8c95f42ba9baa91579fdb5cc9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:53.771795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7212dab8c95f42ba9baa91579fdb5cc9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:53.773757 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:53.830330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7212dab8c95f42ba9baa91579fdb5cc9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:53.831185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7212dab8c95f42ba9baa91579fdb5cc9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:53.843473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7212dab8c95f42ba9baa91579fdb5cc9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:53.850984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7212dab8c95f42ba9baa91579fdb5cc9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:53.859631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7212dab8c95f42ba9baa91579fdb5cc9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:53.866299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1237/2488] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:55:53 2022] DELETE /identity/v3/users/7212dab8c95f42ba9baa91579fdb5cc9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:53.871140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3186a9-3363-4f53-9140-da0aac11d1e8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:53.871643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3186a9-3363-4f53-9140-da0aac11d1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:53.871747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3186a9-3363-4f53-9140-da0aac11d1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:53.889904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dc3186a9-3363-4f53-9140-da0aac11d1e8 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:53.891096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dc3186a9-3363-4f53-9140-da0aac11d1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:53.958000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1252/2489] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:55:53.971121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:53.972782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:54.044154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2e808-9bd1-4d41-b271-74eaf5d4eb6a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:54.049472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ce34d-f163-4117-bfc4-fa13674c4381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:54.050524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ce34d-f163-4117-bfc4-fa13674c4381 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:54.050991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ce34d-f163-4117-bfc4-fa13674c4381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:54.051433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ce34d-f163-4117-bfc4-fa13674c4381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:54.066758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ce34d-f163-4117-bfc4-fa13674c4381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:54.068193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ce34d-f163-4117-bfc4-fa13674c4381 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:54.078008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1238/2490] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:55:54.354446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3186a9-3363-4f53-9140-da0aac11d1e8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:54.355949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dc3186a9-3363-4f53-9140-da0aac11d1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:54.368308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3186a9-3363-4f53-9140-da0aac11d1e8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:54.374023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:54.374930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:54.375205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:54.375505 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] PATH_INFO: `/v3/projects/7ab3e2eec8ae4b338ab5c514d4bcac74` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:54.376918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab3e2eec8ae4b338ab5c514d4bcac74)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:54.378812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:54.430867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab3e2eec8ae4b338ab5c514d4bcac74', 'request_id': 'req-fd74716d-25fc-4b37-a328-dc2d5e662d34'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:54.438173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab3e2eec8ae4b338ab5c514d4bcac74', 'request_id': 'req-fd74716d-25fc-4b37-a328-dc2d5e662d34'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:54.444729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab3e2eec8ae4b338ab5c514d4bcac74', 'request_id': 'req-fd74716d-25fc-4b37-a328-dc2d5e662d34'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:54.455502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] The token cache is being invalidate because project 7ab3e2eec8ae4b338ab5c514d4bcac74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:55:54.455778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:54.456476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:54.458122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1253/2491] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:55:54 2022] DELETE /identity/v3/projects/7ab3e2eec8ae4b338ab5c514d4bcac74 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:54.462229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ce34d-f163-4117-bfc4-fa13674c4381 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:54.472449 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ce34d-f163-4117-bfc4-fa13674c4381 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:54.528857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:54.530092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:54.530416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:54.530791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] PATH_INFO: `/v3/users/1bd178e4ac154682a773be7107e802cf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:54.532686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd178e4ac154682a773be7107e802cf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:54.536008 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:54.589122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd178e4ac154682a773be7107e802cf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:54.589859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd178e4ac154682a773be7107e802cf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:54.602388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd178e4ac154682a773be7107e802cf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:54.609363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd178e4ac154682a773be7107e802cf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:54.620000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f32d7ed5-12eb-4995-99d7-6a043de60174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd178e4ac154682a773be7107e802cf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:54.627797 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1239/2492] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:55:54 2022] DELETE /identity/v3/users/1bd178e4ac154682a773be7107e802cf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:54.917582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:54.919091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:54.932583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74716d-25fc-4b37-a328-dc2d5e662d34 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:54.992227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:54.993122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:54.993408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:54.993681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] PATH_INFO: `/v3/projects/70c44f4979a147a7aad2d936414efb0d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:54.994876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c44f4979a147a7aad2d936414efb0d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:54.996846 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.047103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c44f4979a147a7aad2d936414efb0d', 'request_id': 'req-1d959cdc-b32d-4f55-b65b-5b1cba572204'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:55.055665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c44f4979a147a7aad2d936414efb0d', 'request_id': 'req-1d959cdc-b32d-4f55-b65b-5b1cba572204'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:55.061853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c44f4979a147a7aad2d936414efb0d', 'request_id': 'req-1d959cdc-b32d-4f55-b65b-5b1cba572204'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:55.073165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] The token cache is being invalidate because project 70c44f4979a147a7aad2d936414efb0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:55:55.073447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:55.074156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d959cdc-b32d-4f55-b65b-5b1cba572204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:55:55.075821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1254/2493] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:55:54 2022] DELETE /identity/v3/projects/70c44f4979a147a7aad2d936414efb0d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:55.080981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455ad8d-f862-45e9-8858-bb6ac28747e0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.081482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455ad8d-f862-45e9-8858-bb6ac28747e0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.081482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455ad8d-f862-45e9-8858-bb6ac28747e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.100729 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a455ad8d-f862-45e9-8858-bb6ac28747e0 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:55.101914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a455ad8d-f862-45e9-8858-bb6ac28747e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:55.174638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1240/2494] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:55:55.187844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731525df-dcdc-4112-94f7-93abf38e3046 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.188131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731525df-dcdc-4112-94f7-93abf38e3046 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.188416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731525df-dcdc-4112-94f7-93abf38e3046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.209105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-731525df-dcdc-4112-94f7-93abf38e3046 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:55.210308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-731525df-dcdc-4112-94f7-93abf38e3046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:55.218875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1255/2495] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:55:55.223315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455ad8d-f862-45e9-8858-bb6ac28747e0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.233436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455ad8d-f862-45e9-8858-bb6ac28747e0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:55.239310 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475c15b-98a3-4f63-becb-3be978badc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:55.239960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475c15b-98a3-4f63-becb-3be978badc70 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.240237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475c15b-98a3-4f63-becb-3be978badc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.240512 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475c15b-98a3-4f63-becb-3be978badc70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.241249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475c15b-98a3-4f63-becb-3be978badc70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:55:55.241564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475c15b-98a3-4f63-becb-3be978badc70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:55.243176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475c15b-98a3-4f63-becb-3be978badc70 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.256188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1241/2496] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:55:55.261200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731525df-dcdc-4112-94f7-93abf38e3046 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.273623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731525df-dcdc-4112-94f7-93abf38e3046 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:55.280680 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6633fe-e275-4dd4-87d6-39f863e11166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:55.281351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6633fe-e275-4dd4-87d6-39f863e11166 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.281628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6633fe-e275-4dd4-87d6-39f863e11166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.281886 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6633fe-e275-4dd4-87d6-39f863e11166 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.282584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6633fe-e275-4dd4-87d6-39f863e11166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:55.284577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6633fe-e275-4dd4-87d6-39f863e11166 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.320272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1256/2497] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:55:55 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:55:55.325397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475c15b-98a3-4f63-becb-3be978badc70 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.336645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475c15b-98a3-4f63-becb-3be978badc70 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:55.404907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda18b0-031c-4a6b-bdc3-635c5045ca0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:55.405662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda18b0-031c-4a6b-bdc3-635c5045ca0e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.405945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda18b0-031c-4a6b-bdc3-635c5045ca0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.406220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda18b0-031c-4a6b-bdc3-635c5045ca0e admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.406906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda18b0-031c-4a6b-bdc3-635c5045ca0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:55.408862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda18b0-031c-4a6b-bdc3-635c5045ca0e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.431059 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1242/2498] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:55:55 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:55:55.435690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6633fe-e275-4dd4-87d6-39f863e11166 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.446725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6633fe-e275-4dd4-87d6-39f863e11166 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:55.451675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5bbb2-de88-4acc-8090-3ca6cd8b5493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:55.452326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5bbb2-de88-4acc-8090-3ca6cd8b5493 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.452631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5bbb2-de88-4acc-8090-3ca6cd8b5493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.452897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5bbb2-de88-4acc-8090-3ca6cd8b5493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.453609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5bbb2-de88-4acc-8090-3ca6cd8b5493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:55.455191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5bbb2-de88-4acc-8090-3ca6cd8b5493 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.465610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1257/2499] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:55:55.469778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda18b0-031c-4a6b-bdc3-635c5045ca0e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.481839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda18b0-031c-4a6b-bdc3-635c5045ca0e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:55.487596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a743c0-4ff3-4bcc-aaa9-70c4c8329df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:55.488350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a743c0-4ff3-4bcc-aaa9-70c4c8329df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.488681 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a743c0-4ff3-4bcc-aaa9-70c4c8329df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.488954 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a743c0-4ff3-4bcc-aaa9-70c4c8329df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.489662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a743c0-4ff3-4bcc-aaa9-70c4c8329df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:55.491257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a743c0-4ff3-4bcc-aaa9-70c4c8329df0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.501690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1243/2500] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:55:55.507311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5bbb2-de88-4acc-8090-3ca6cd8b5493 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.522563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5bbb2-de88-4acc-8090-3ca6cd8b5493 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:55.531362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd333-95c9-4d1a-b8c9-b951a0171b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:55.532768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd333-95c9-4d1a-b8c9-b951a0171b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.533183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd333-95c9-4d1a-b8c9-b951a0171b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.533626 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd333-95c9-4d1a-b8c9-b951a0171b19 admin admin] PATH_INFO: `/v3/projects/538b43e4dfd842e79d514a3f1bd829e1/users/7908c344fd274c2a8c507cf9efd84411/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.551880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abd333-95c9-4d1a-b8c9-b951a0171b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538b43e4dfd842e79d514a3f1bd829e1, user_id=7908c344fd274c2a8c507cf9efd84411, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:55.554572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abd333-95c9-4d1a-b8c9-b951a0171b19 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.575892 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1258/2501] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:55:55 2022] PUT /identity/v3/projects/538b43e4dfd842e79d514a3f1bd829e1/users/7908c344fd274c2a8c507cf9efd84411/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:55.591941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf570c-08f6-44f3-8f1e-9ad5d60f4aa7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.592320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf570c-08f6-44f3-8f1e-9ad5d60f4aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.592735 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf570c-08f6-44f3-8f1e-9ad5d60f4aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.621616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8bdf570c-08f6-44f3-8f1e-9ad5d60f4aa7 None None] MFA Rules not processed for user `7908c344fd274c2a8c507cf9efd84411`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:55.623004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8bdf570c-08f6-44f3-8f1e-9ad5d60f4aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:55.685161 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1244/2502] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:55:55.690143 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd333-95c9-4d1a-b8c9-b951a0171b19 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.753317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd333-95c9-4d1a-b8c9-b951a0171b19 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:55.758163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56555d7e-0588-41ce-ab14-a4ea48ba614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:55.758824 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56555d7e-0588-41ce-ab14-a4ea48ba614d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.759123 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56555d7e-0588-41ce-ab14-a4ea48ba614d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.759384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56555d7e-0588-41ce-ab14-a4ea48ba614d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.760078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56555d7e-0588-41ce-ab14-a4ea48ba614d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:55.761994 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56555d7e-0588-41ce-ab14-a4ea48ba614d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.791105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1259/2503] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:55:55 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:55:55.792230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf570c-08f6-44f3-8f1e-9ad5d60f4aa7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.802795 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf570c-08f6-44f3-8f1e-9ad5d60f4aa7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:55.859184 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafebe0-6ea9-4a8b-9050-534da1b43890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:55.859860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafebe0-6ea9-4a8b-9050-534da1b43890 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.860136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafebe0-6ea9-4a8b-9050-534da1b43890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.860416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafebe0-6ea9-4a8b-9050-534da1b43890 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.861137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafebe0-6ea9-4a8b-9050-534da1b43890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:55.863187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafebe0-6ea9-4a8b-9050-534da1b43890 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.883496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1245/2504] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:55:55 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:55:55.887619 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56555d7e-0588-41ce-ab14-a4ea48ba614d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.897984 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56555d7e-0588-41ce-ab14-a4ea48ba614d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:55.902720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d34b6-b977-42fd-8608-300e93151566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:55.903450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d34b6-b977-42fd-8608-300e93151566 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.903795 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d34b6-b977-42fd-8608-300e93151566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.904134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d34b6-b977-42fd-8608-300e93151566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.904967 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d34b6-b977-42fd-8608-300e93151566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:55.906622 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d34b6-b977-42fd-8608-300e93151566 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.917203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1260/2505] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:55:55.921065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafebe0-6ea9-4a8b-9050-534da1b43890 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.931626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafebe0-6ea9-4a8b-9050-534da1b43890 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:55.937028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d4efb-3132-4ea4-8327-5264388abd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:55.937944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d4efb-3132-4ea4-8327-5264388abd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:55.938448 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d4efb-3132-4ea4-8327-5264388abd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:55.938722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d4efb-3132-4ea4-8327-5264388abd5a admin admin] PATH_INFO: `/v3/projects/3bddf41a687f4ffdaaebfa2d4764a898/users/cb17cc4380a5461484ad8d8d8ee90ab7/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:55.956340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d4efb-3132-4ea4-8327-5264388abd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bddf41a687f4ffdaaebfa2d4764a898, user_id=cb17cc4380a5461484ad8d8d8ee90ab7, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:55.959101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d4efb-3132-4ea4-8327-5264388abd5a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:55.973617 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1246/2506] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:55:55 2022] PUT /identity/v3/projects/3bddf41a687f4ffdaaebfa2d4764a898/users/cb17cc4380a5461484ad8d8d8ee90ab7/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:55.977572 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d34b6-b977-42fd-8608-300e93151566 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:55.988140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d34b6-b977-42fd-8608-300e93151566 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:56.047339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcbd89-e217-44a3-afb5-c0702acd5041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:56.048108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcbd89-e217-44a3-afb5-c0702acd5041 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:56.048454 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcbd89-e217-44a3-afb5-c0702acd5041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:56.048786 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcbd89-e217-44a3-afb5-c0702acd5041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:56.049623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcbd89-e217-44a3-afb5-c0702acd5041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:56.051429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcbd89-e217-44a3-afb5-c0702acd5041 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:56.061653 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1261/2507] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:55:56.066364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d4efb-3132-4ea4-8327-5264388abd5a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:56.077133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d4efb-3132-4ea4-8327-5264388abd5a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:56.081763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fe716-5711-488c-9650-4577593860fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:56.082859 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe716-5711-488c-9650-4577593860fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:56.083136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe716-5711-488c-9650-4577593860fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:56.083418 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fe716-5711-488c-9650-4577593860fe admin admin] PATH_INFO: `/v3/projects/3bddf41a687f4ffdaaebfa2d4764a898/users/cb17cc4380a5461484ad8d8d8ee90ab7/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:56.085370 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fe716-5711-488c-9650-4577593860fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bddf41a687f4ffdaaebfa2d4764a898, user_id=cb17cc4380a5461484ad8d8d8ee90ab7, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:56.088032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fe716-5711-488c-9650-4577593860fe admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:56.099759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1247/2508] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:55:56 2022] PUT /identity/v3/projects/3bddf41a687f4ffdaaebfa2d4764a898/users/cb17cc4380a5461484ad8d8d8ee90ab7/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:55:56.110159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c755e64-96a3-4f37-a67c-1c60f8b005a0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:56.110389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c755e64-96a3-4f37-a67c-1c60f8b005a0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:56.110730 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c755e64-96a3-4f37-a67c-1c60f8b005a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:56.140224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4c755e64-96a3-4f37-a67c-1c60f8b005a0 None None] MFA Rules not processed for user `cb17cc4380a5461484ad8d8d8ee90ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:56.141822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4c755e64-96a3-4f37-a67c-1c60f8b005a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:56.211983 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1262/2509] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:55:56.224671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fe716-5711-488c-9650-4577593860fe admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:55:56.226586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fe7fe716-5711-488c-9650-4577593860fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:56.292232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fe716-5711-488c-9650-4577593860fe admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:55:56.295937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f390f10-5357-4b3f-bd18-64e1d7189406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:55:56.296615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f390f10-5357-4b3f-bd18-64e1d7189406 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:56.296896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f390f10-5357-4b3f-bd18-64e1d7189406 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:56.297162 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f390f10-5357-4b3f-bd18-64e1d7189406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:56.352029 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f390f10-5357-4b3f-bd18-64e1d7189406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:55:56.353496 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f390f10-5357-4b3f-bd18-64e1d7189406 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:55:56.362721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1248/2510] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:55:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:55:57.201638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86207-15ff-47c0-9e2a-a1902f9fe219 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:55:57.202165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86207-15ff-47c0-9e2a-a1902f9fe219 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:55:57.202165 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade86207-15ff-47c0-9e2a-a1902f9fe219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:55:57.219784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ade86207-15ff-47c0-9e2a-a1902f9fe219 None None] MFA Rules not processed for user `cc76dcb94459489f965c9205f2d461cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:55:57.220998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ade86207-15ff-47c0-9e2a-a1902f9fe219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:55:57.280256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1263/2511] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:55:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:02.665337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f390f10-5357-4b3f-bd18-64e1d7189406 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:02.667666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3f390f10-5357-4b3f-bd18-64e1d7189406 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:02.703380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f390f10-5357-4b3f-bd18-64e1d7189406 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:02.707744 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3b47a-984b-4913-9ff8-e0ae45083237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:02.708371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3b47a-984b-4913-9ff8-e0ae45083237 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:02.708687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3b47a-984b-4913-9ff8-e0ae45083237 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:02.708953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3b47a-984b-4913-9ff8-e0ae45083237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:02.719572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3b47a-984b-4913-9ff8-e0ae45083237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:02.721066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3b47a-984b-4913-9ff8-e0ae45083237 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:02.731595 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1249/2512] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:07.807142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade86207-15ff-47c0-9e2a-a1902f9fe219 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:07.872820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade86207-15ff-47c0-9e2a-a1902f9fe219 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:07.877510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:07.878479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:07.878688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:07.878954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] PATH_INFO: `/v3/users/7908c344fd274c2a8c507cf9efd84411` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:07.880113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7908c344fd274c2a8c507cf9efd84411)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:07.882098 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:07.927838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7908c344fd274c2a8c507cf9efd84411'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:07.928540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7908c344fd274c2a8c507cf9efd84411'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:07.941243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7908c344fd274c2a8c507cf9efd84411'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:07.950623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7908c344fd274c2a8c507cf9efd84411'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:07.959429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7908c344fd274c2a8c507cf9efd84411'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:07.966809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1264/2513] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:56:07 2022] DELETE /identity/v3/users/7908c344fd274c2a8c507cf9efd84411 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:07.971637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5b44e-be65-4a00-806d-91433163d325 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:07.972183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5b44e-be65-4a00-806d-91433163d325 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:07.972183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5b44e-be65-4a00-806d-91433163d325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:07.992555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f0a5b44e-be65-4a00-806d-91433163d325 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:07.993805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f0a5b44e-be65-4a00-806d-91433163d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:08.056877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1250/2514] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:08.067404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:08.068939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:08.132167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f42a54-2cfd-4abb-ad5a-22d3bd9a6b00 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:08.136833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324c35d-e228-40b8-9b9b-4fb21da52539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:08.137566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324c35d-e228-40b8-9b9b-4fb21da52539 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:08.137914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324c35d-e228-40b8-9b9b-4fb21da52539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:08.138187 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324c35d-e228-40b8-9b9b-4fb21da52539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:08.148319 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324c35d-e228-40b8-9b9b-4fb21da52539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:08.149788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324c35d-e228-40b8-9b9b-4fb21da52539 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:08.160138 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1265/2515] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:08.449529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5b44e-be65-4a00-806d-91433163d325 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:08.451615 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f0a5b44e-be65-4a00-806d-91433163d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:08.467221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5b44e-be65-4a00-806d-91433163d325 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:08.472305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:08.474404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:08.474741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:08.474936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] PATH_INFO: `/v3/projects/538b43e4dfd842e79d514a3f1bd829e1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:08.476229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] RBAC: Authorizing `identity:delete_project(project_id=538b43e4dfd842e79d514a3f1bd829e1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:08.478227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:08.524433 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538b43e4dfd842e79d514a3f1bd829e1', 'request_id': 'req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:08.531712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538b43e4dfd842e79d514a3f1bd829e1', 'request_id': 'req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:08.538747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538b43e4dfd842e79d514a3f1bd829e1', 'request_id': 'req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:08.549252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] The token cache is being invalidate because project 538b43e4dfd842e79d514a3f1bd829e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:56:08.549531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:08.550201 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:08.551850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1251/2516] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:56:08 2022] DELETE /identity/v3/projects/538b43e4dfd842e79d514a3f1bd829e1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:56:08.556061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324c35d-e228-40b8-9b9b-4fb21da52539 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:08.566027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324c35d-e228-40b8-9b9b-4fb21da52539 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:08.620034 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:08.620944 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:08.621222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:08.621459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] PATH_INFO: `/v3/users/cb17cc4380a5461484ad8d8d8ee90ab7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:08.622909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb17cc4380a5461484ad8d8d8ee90ab7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:08.624865 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:08.675278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb17cc4380a5461484ad8d8d8ee90ab7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:08.675916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb17cc4380a5461484ad8d8d8ee90ab7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:08.687742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb17cc4380a5461484ad8d8d8ee90ab7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:08.695748 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb17cc4380a5461484ad8d8d8ee90ab7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:08.703103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a50422e2-64cc-41bf-a21b-bab5a31484ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb17cc4380a5461484ad8d8d8ee90ab7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:08.709351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1266/2517] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:56:08 2022] DELETE /identity/v3/users/cb17cc4380a5461484ad8d8d8ee90ab7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:08.986605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:08.988094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:09.004059 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b905b4-1a6a-4a4d-bb9b-1256907ba2be admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:09.060956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:09.061847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.062097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.062359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] PATH_INFO: `/v3/projects/3bddf41a687f4ffdaaebfa2d4764a898` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.063514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bddf41a687f4ffdaaebfa2d4764a898)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:09.065813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:09.112262 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bddf41a687f4ffdaaebfa2d4764a898', 'request_id': 'req-af182691-86a9-4c2f-9c0d-b22b6cac5cab'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:09.120414 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bddf41a687f4ffdaaebfa2d4764a898', 'request_id': 'req-af182691-86a9-4c2f-9c0d-b22b6cac5cab'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:09.126539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bddf41a687f4ffdaaebfa2d4764a898', 'request_id': 'req-af182691-86a9-4c2f-9c0d-b22b6cac5cab'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:09.137802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] The token cache is being invalidate because project 3bddf41a687f4ffdaaebfa2d4764a898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:56:09.138090 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:09.138713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-af182691-86a9-4c2f-9c0d-b22b6cac5cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:09.140292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1252/2518] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:56:08 2022] DELETE /identity/v3/projects/3bddf41a687f4ffdaaebfa2d4764a898 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:09.146588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230de2a-219b-488a-90b1-5f936378acfa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.147063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230de2a-219b-488a-90b1-5f936378acfa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.147158 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230de2a-219b-488a-90b1-5f936378acfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.165711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d230de2a-219b-488a-90b1-5f936378acfa None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:09.166918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d230de2a-219b-488a-90b1-5f936378acfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:09.228286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1267/2519] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:09.238728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214910a0-526f-40b5-8a40-4afcea95b821 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.239070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214910a0-526f-40b5-8a40-4afcea95b821 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.239513 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214910a0-526f-40b5-8a40-4afcea95b821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.259849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-214910a0-526f-40b5-8a40-4afcea95b821 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:09.261094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-214910a0-526f-40b5-8a40-4afcea95b821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:09.270828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1253/2520] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:09.275968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230de2a-219b-488a-90b1-5f936378acfa None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:09.285601 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230de2a-219b-488a-90b1-5f936378acfa None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:09.290667 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b0e54-ca27-4def-b2c2-37f4c09ddab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:09.291315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b0e54-ca27-4def-b2c2-37f4c09ddab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.291593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b0e54-ca27-4def-b2c2-37f4c09ddab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.291866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b0e54-ca27-4def-b2c2-37f4c09ddab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.292618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b0e54-ca27-4def-b2c2-37f4c09ddab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:56:09.292925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b0e54-ca27-4def-b2c2-37f4c09ddab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:09.294522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b0e54-ca27-4def-b2c2-37f4c09ddab8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:09.306256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1268/2521] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:56:09.310973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214910a0-526f-40b5-8a40-4afcea95b821 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:09.321048 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214910a0-526f-40b5-8a40-4afcea95b821 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:09.326981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e7c69-f53e-4173-b42e-1b0aa00bbdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:09.327654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e7c69-f53e-4173-b42e-1b0aa00bbdd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.327916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e7c69-f53e-4173-b42e-1b0aa00bbdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.328189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e7c69-f53e-4173-b42e-1b0aa00bbdd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.328920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e7c69-f53e-4173-b42e-1b0aa00bbdd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:09.330828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e7c69-f53e-4173-b42e-1b0aa00bbdd2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:09.357815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1254/2522] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:56:09 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:56:09.362253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b0e54-ca27-4def-b2c2-37f4c09ddab8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:09.373410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b0e54-ca27-4def-b2c2-37f4c09ddab8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:09.436849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f6192-7e05-41c3-8af9-f942593c987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:09.437512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f6192-7e05-41c3-8af9-f942593c987a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.437787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f6192-7e05-41c3-8af9-f942593c987a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.438055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f6192-7e05-41c3-8af9-f942593c987a admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.439097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f6192-7e05-41c3-8af9-f942593c987a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:09.441066 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f6192-7e05-41c3-8af9-f942593c987a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:09.461878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1269/2523] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:56:09 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:56:09.466287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e7c69-f53e-4173-b42e-1b0aa00bbdd2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:09.477292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e7c69-f53e-4173-b42e-1b0aa00bbdd2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:09.482761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0c336-a911-4068-815a-4937501b8f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:09.483366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0c336-a911-4068-815a-4937501b8f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.483645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0c336-a911-4068-815a-4937501b8f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.483915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0c336-a911-4068-815a-4937501b8f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.484680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0c336-a911-4068-815a-4937501b8f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:09.486304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0c336-a911-4068-815a-4937501b8f4d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:09.497085 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1255/2524] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:09.501131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f6192-7e05-41c3-8af9-f942593c987a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:09.511558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f6192-7e05-41c3-8af9-f942593c987a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:09.516637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd8d57-8fdc-4a1b-ae33-a9c1bab81e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:09.517927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd8d57-8fdc-4a1b-ae33-a9c1bab81e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.518198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd8d57-8fdc-4a1b-ae33-a9c1bab81e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.518470 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd8d57-8fdc-4a1b-ae33-a9c1bab81e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.519201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd8d57-8fdc-4a1b-ae33-a9c1bab81e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:09.520826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd8d57-8fdc-4a1b-ae33-a9c1bab81e02 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:09.531556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1270/2525] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:09.535671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0c336-a911-4068-815a-4937501b8f4d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:09.546465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0c336-a911-4068-815a-4937501b8f4d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:09.552240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910009c-251b-4e2f-b248-c3224d5cb25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:09.553191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910009c-251b-4e2f-b248-c3224d5cb25e admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.553464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910009c-251b-4e2f-b248-c3224d5cb25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.553730 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910009c-251b-4e2f-b248-c3224d5cb25e admin admin] PATH_INFO: `/v3/projects/1adb5e0f25404c1b981269f9ca1eac5a/users/c1f4bdfc63c94ddfab870a781070e0d9/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.568938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e910009c-251b-4e2f-b248-c3224d5cb25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adb5e0f25404c1b981269f9ca1eac5a, user_id=c1f4bdfc63c94ddfab870a781070e0d9, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:09.571905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e910009c-251b-4e2f-b248-c3224d5cb25e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:09.586231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1256/2526] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:56:09 2022] PUT /identity/v3/projects/1adb5e0f25404c1b981269f9ca1eac5a/users/c1f4bdfc63c94ddfab870a781070e0d9/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:09.597686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c748cd-eae2-4e4d-9499-3279d71b04df None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.598017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c748cd-eae2-4e4d-9499-3279d71b04df None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.598328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c748cd-eae2-4e4d-9499-3279d71b04df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.626535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c7c748cd-eae2-4e4d-9499-3279d71b04df None None] MFA Rules not processed for user `c1f4bdfc63c94ddfab870a781070e0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:09.627828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7c748cd-eae2-4e4d-9499-3279d71b04df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:09.694937 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1271/2527] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:09.706741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910009c-251b-4e2f-b248-c3224d5cb25e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:09.708624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e910009c-251b-4e2f-b248-c3224d5cb25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:09.763782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910009c-251b-4e2f-b248-c3224d5cb25e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:09.767261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace316d-a0ad-4ced-8bab-f46b8f993ae3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:09.767899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace316d-a0ad-4ced-8bab-f46b8f993ae3 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.768208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace316d-a0ad-4ced-8bab-f46b8f993ae3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.768471 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace316d-a0ad-4ced-8bab-f46b8f993ae3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.776464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace316d-a0ad-4ced-8bab-f46b8f993ae3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:09.777930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace316d-a0ad-4ced-8bab-f46b8f993ae3 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:09.786204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1257/2528] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:09.805843 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c748cd-eae2-4e4d-9499-3279d71b04df None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:09.867958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c748cd-eae2-4e4d-9499-3279d71b04df None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:09.873684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:09.874575 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.874858 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.875130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] PATH_INFO: `/v3/users/c1f4bdfc63c94ddfab870a781070e0d9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:09.876447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f4bdfc63c94ddfab870a781070e0d9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:09.878559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:09.940370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f4bdfc63c94ddfab870a781070e0d9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:09.941479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f4bdfc63c94ddfab870a781070e0d9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:09.957202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f4bdfc63c94ddfab870a781070e0d9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:09.966394 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f4bdfc63c94ddfab870a781070e0d9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:09.973359 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f4bdfc63c94ddfab870a781070e0d9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:09.979371 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1272/2529] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:56:09 2022] DELETE /identity/v3/users/c1f4bdfc63c94ddfab870a781070e0d9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:09.984067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be40ea2-5506-4128-bf3c-cbaf5f82b058 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:09.984357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be40ea2-5506-4128-bf3c-cbaf5f82b058 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:09.984673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be40ea2-5506-4128-bf3c-cbaf5f82b058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.002385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0be40ea2-5506-4128-bf3c-cbaf5f82b058 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:10.003559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0be40ea2-5506-4128-bf3c-cbaf5f82b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:10.064189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1258/2530] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:10.074146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:10.075600 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:10.139253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15b40b-d18e-4746-9af8-df3e625a5fd9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:10.144028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf740e-2e00-4e2b-befe-517c6f6545be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:10.144713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf740e-2e00-4e2b-befe-517c6f6545be service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.144993 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf740e-2e00-4e2b-befe-517c6f6545be service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.145268 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf740e-2e00-4e2b-befe-517c6f6545be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.154835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf740e-2e00-4e2b-befe-517c6f6545be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:10.156263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf740e-2e00-4e2b-befe-517c6f6545be service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:10.165776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1273/2531] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:10.435225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be40ea2-5506-4128-bf3c-cbaf5f82b058 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:10.436698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0be40ea2-5506-4128-bf3c-cbaf5f82b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:10.452434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be40ea2-5506-4128-bf3c-cbaf5f82b058 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:10.458137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:10.459052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.459334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.459647 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] PATH_INFO: `/v3/projects/1adb5e0f25404c1b981269f9ca1eac5a` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.461740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1adb5e0f25404c1b981269f9ca1eac5a)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:10.463977 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:10.510303 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1adb5e0f25404c1b981269f9ca1eac5a', 'request_id': 'req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:10.521479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1adb5e0f25404c1b981269f9ca1eac5a', 'request_id': 'req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:10.531097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1adb5e0f25404c1b981269f9ca1eac5a', 'request_id': 'req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:10.542446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] The token cache is being invalidate because project 1adb5e0f25404c1b981269f9ca1eac5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:56:10.542833 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:10.543482 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4ffb81d7-f23b-4e1f-888b-e064dec9ef54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:10.545199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1259/2532] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:56:10 2022] DELETE /identity/v3/projects/1adb5e0f25404c1b981269f9ca1eac5a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:10.550449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d62f9-3762-4c47-96e5-4466282040ac None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.550737 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d62f9-3762-4c47-96e5-4466282040ac None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.551010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d62f9-3762-4c47-96e5-4466282040ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.568915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3a4d62f9-3762-4c47-96e5-4466282040ac None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:10.570129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3a4d62f9-3762-4c47-96e5-4466282040ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:10.631031 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1274/2533] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:10.639294 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262dd565-dec7-453b-be58-eefaa215e8bb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.639587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262dd565-dec7-453b-be58-eefaa215e8bb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.639865 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262dd565-dec7-453b-be58-eefaa215e8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.660129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-262dd565-dec7-453b-be58-eefaa215e8bb None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:10.661377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-262dd565-dec7-453b-be58-eefaa215e8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:10.670463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1260/2534] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:10.674961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d62f9-3762-4c47-96e5-4466282040ac None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:10.685408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d62f9-3762-4c47-96e5-4466282040ac None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:10.691602 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155cffb-127a-4760-878e-6aef53db88f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:10.692268 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155cffb-127a-4760-878e-6aef53db88f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.692570 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155cffb-127a-4760-878e-6aef53db88f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.692838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155cffb-127a-4760-878e-6aef53db88f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.693582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155cffb-127a-4760-878e-6aef53db88f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:56:10.693867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155cffb-127a-4760-878e-6aef53db88f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:10.695502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155cffb-127a-4760-878e-6aef53db88f3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:10.707048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1275/2535] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:56:10.711628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262dd565-dec7-453b-be58-eefaa215e8bb None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:10.721313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262dd565-dec7-453b-be58-eefaa215e8bb None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:10.726856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cb81a-711c-4234-9f2d-782e6d8c4d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:10.727507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cb81a-711c-4234-9f2d-782e6d8c4d96 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.727783 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cb81a-711c-4234-9f2d-782e6d8c4d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.728048 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cb81a-711c-4234-9f2d-782e6d8c4d96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.728766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cb81a-711c-4234-9f2d-782e6d8c4d96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:10.730662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cb81a-711c-4234-9f2d-782e6d8c4d96 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:10.759053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1261/2536] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:56:10 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:56:10.763285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155cffb-127a-4760-878e-6aef53db88f3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:10.773624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155cffb-127a-4760-878e-6aef53db88f3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:10.836003 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee47c5e-6d7b-4026-a538-6cb7dd4d9e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:10.836710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee47c5e-6d7b-4026-a538-6cb7dd4d9e7a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.836985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee47c5e-6d7b-4026-a538-6cb7dd4d9e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.837257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee47c5e-6d7b-4026-a538-6cb7dd4d9e7a admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.837893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee47c5e-6d7b-4026-a538-6cb7dd4d9e7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:10.839814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee47c5e-6d7b-4026-a538-6cb7dd4d9e7a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:10.860358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1276/2537] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:56:10 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:56:10.864514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cb81a-711c-4234-9f2d-782e6d8c4d96 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:10.875257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cb81a-711c-4234-9f2d-782e6d8c4d96 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:10.880808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10220dc6-7cae-4a1f-97be-b7d11c832048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:10.881613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10220dc6-7cae-4a1f-97be-b7d11c832048 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.881899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10220dc6-7cae-4a1f-97be-b7d11c832048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.882175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10220dc6-7cae-4a1f-97be-b7d11c832048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.883465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10220dc6-7cae-4a1f-97be-b7d11c832048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:10.885048 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10220dc6-7cae-4a1f-97be-b7d11c832048 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:10.896850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1262/2538] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:10.901303 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee47c5e-6d7b-4026-a538-6cb7dd4d9e7a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:10.911379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee47c5e-6d7b-4026-a538-6cb7dd4d9e7a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:10.916899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e77b1-8beb-4f61-b27f-0fcf4ae7c811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:10.917558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e77b1-8beb-4f61-b27f-0fcf4ae7c811 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.917844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e77b1-8beb-4f61-b27f-0fcf4ae7c811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.918125 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e77b1-8beb-4f61-b27f-0fcf4ae7c811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.918855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e77b1-8beb-4f61-b27f-0fcf4ae7c811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:10.920465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e77b1-8beb-4f61-b27f-0fcf4ae7c811 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:10.930627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1277/2539] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:10.934596 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10220dc6-7cae-4a1f-97be-b7d11c832048 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:10.944813 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10220dc6-7cae-4a1f-97be-b7d11c832048 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:10.950514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4ba91-4475-4bc2-99ab-93581505dcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:10.951388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4ba91-4475-4bc2-99ab-93581505dcae admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.951669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4ba91-4475-4bc2-99ab-93581505dcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.951940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4ba91-4475-4bc2-99ab-93581505dcae admin admin] PATH_INFO: `/v3/projects/5cb132017e094f3a8fee00bab6dad6e4/users/a357bbcbd2c04ad8a63e3a71de220e13/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:10.967142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4ba91-4475-4bc2-99ab-93581505dcae admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb132017e094f3a8fee00bab6dad6e4, user_id=a357bbcbd2c04ad8a63e3a71de220e13, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:10.969867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4ba91-4475-4bc2-99ab-93581505dcae admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:10.983929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1263/2540] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:56:10 2022] PUT /identity/v3/projects/5cb132017e094f3a8fee00bab6dad6e4/users/a357bbcbd2c04ad8a63e3a71de220e13/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:56:10.995180 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ece79-d163-4026-b9ed-73d05c912268 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:10.995464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ece79-d163-4026-b9ed-73d05c912268 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:10.995730 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ece79-d163-4026-b9ed-73d05c912268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:11.022610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9e2ece79-d163-4026-b9ed-73d05c912268 None None] MFA Rules not processed for user `a357bbcbd2c04ad8a63e3a71de220e13`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:11.024032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9e2ece79-d163-4026-b9ed-73d05c912268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:11.089243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1278/2541] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:11.102785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4ba91-4475-4bc2-99ab-93581505dcae admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:11.104679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-29f4ba91-4475-4bc2-99ab-93581505dcae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:11.162822 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4ba91-4475-4bc2-99ab-93581505dcae admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:11.166698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948a939-b975-4912-bc44-90ce61731f42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:11.167336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948a939-b975-4912-bc44-90ce61731f42 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:11.167610 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948a939-b975-4912-bc44-90ce61731f42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:11.167882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948a939-b975-4912-bc44-90ce61731f42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:11.181500 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948a939-b975-4912-bc44-90ce61731f42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:11.183639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948a939-b975-4912-bc44-90ce61731f42 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:11.197231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1264/2542] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:56:39.282930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ece79-d163-4026-b9ed-73d05c912268 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:39.348820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ece79-d163-4026-b9ed-73d05c912268 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:39.354282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:39.355157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:39.355432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:39.355702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] PATH_INFO: `/v3/users/a357bbcbd2c04ad8a63e3a71de220e13` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:39.356946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a357bbcbd2c04ad8a63e3a71de220e13)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:39.358837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:39.407334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a357bbcbd2c04ad8a63e3a71de220e13'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:39.408241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a357bbcbd2c04ad8a63e3a71de220e13'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:39.422173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a357bbcbd2c04ad8a63e3a71de220e13'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:39.433320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a357bbcbd2c04ad8a63e3a71de220e13'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:39.440641 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a357bbcbd2c04ad8a63e3a71de220e13'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:39.446574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1279/2543] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:56:39 2022] DELETE /identity/v3/users/a357bbcbd2c04ad8a63e3a71de220e13 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:39.451272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc23d6f-d7a4-497b-83f6-4f7594780a78 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:39.451766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc23d6f-d7a4-497b-83f6-4f7594780a78 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:39.451871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc23d6f-d7a4-497b-83f6-4f7594780a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:39.470046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0bc23d6f-d7a4-497b-83f6-4f7594780a78 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:39.471236 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0bc23d6f-d7a4-497b-83f6-4f7594780a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:39.531477 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1265/2544] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:39.543651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:39.546273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:39.614727 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a29f6-baf1-4282-86f1-4e964ab9b09f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:39.620070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8724a60-ce1c-42a5-b48b-90dd1460da16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:39.620787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8724a60-ce1c-42a5-b48b-90dd1460da16 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:39.621006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8724a60-ce1c-42a5-b48b-90dd1460da16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:39.621282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8724a60-ce1c-42a5-b48b-90dd1460da16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:39.632526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8724a60-ce1c-42a5-b48b-90dd1460da16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:39.633991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8724a60-ce1c-42a5-b48b-90dd1460da16 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:39.643807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1280/2545] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:39.922666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc23d6f-d7a4-497b-83f6-4f7594780a78 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:39.924126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0bc23d6f-d7a4-497b-83f6-4f7594780a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:39.935716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc23d6f-d7a4-497b-83f6-4f7594780a78 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:39.941162 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:39.942097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:39.942380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:39.942644 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] PATH_INFO: `/v3/projects/5cb132017e094f3a8fee00bab6dad6e4` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:39.943827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cb132017e094f3a8fee00bab6dad6e4)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:39.945905 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:39.996538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cb132017e094f3a8fee00bab6dad6e4', 'request_id': 'req-470cf32b-2392-456b-a0dd-73c801029d43'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:40.003819 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cb132017e094f3a8fee00bab6dad6e4', 'request_id': 'req-470cf32b-2392-456b-a0dd-73c801029d43'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:40.010148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cb132017e094f3a8fee00bab6dad6e4', 'request_id': 'req-470cf32b-2392-456b-a0dd-73c801029d43'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:40.019968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] The token cache is being invalidate because project 5cb132017e094f3a8fee00bab6dad6e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:56:40.020302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:40.020991 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-470cf32b-2392-456b-a0dd-73c801029d43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:40.022643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1266/2546] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:56:39 2022] DELETE /identity/v3/projects/5cb132017e094f3a8fee00bab6dad6e4 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:40.027868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c68c7b-7442-4475-b617-9c1f4f793c10 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.028152 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c68c7b-7442-4475-b617-9c1f4f793c10 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.028452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c68c7b-7442-4475-b617-9c1f4f793c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.047172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-85c68c7b-7442-4475-b617-9c1f4f793c10 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:40.048378 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-85c68c7b-7442-4475-b617-9c1f4f793c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:40.116317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1281/2547] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:40.124416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63409d-ab79-4deb-acda-cb2af975fdc9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.124737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63409d-ab79-4deb-acda-cb2af975fdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.125003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63409d-ab79-4deb-acda-cb2af975fdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.143543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ab63409d-ab79-4deb-acda-cb2af975fdc9 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:40.144755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ab63409d-ab79-4deb-acda-cb2af975fdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:40.153021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1267/2548] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:40.157488 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c68c7b-7442-4475-b617-9c1f4f793c10 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.167478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c68c7b-7442-4475-b617-9c1f4f793c10 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.173303 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67ca53-54cc-4535-b788-36ebe90d5c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.173989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67ca53-54cc-4535-b788-36ebe90d5c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.174273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67ca53-54cc-4535-b788-36ebe90d5c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.174544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67ca53-54cc-4535-b788-36ebe90d5c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.175263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67ca53-54cc-4535-b788-36ebe90d5c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:56:40.175571 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67ca53-54cc-4535-b788-36ebe90d5c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.177230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67ca53-54cc-4535-b788-36ebe90d5c96 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.189491 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1282/2549] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:56:40.193786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63409d-ab79-4deb-acda-cb2af975fdc9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.202850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63409d-ab79-4deb-acda-cb2af975fdc9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.208209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aecb6-63b5-4897-bcb2-f5e602c66244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.208879 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aecb6-63b5-4897-bcb2-f5e602c66244 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.209156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aecb6-63b5-4897-bcb2-f5e602c66244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.209422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aecb6-63b5-4897-bcb2-f5e602c66244 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.210217 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aecb6-63b5-4897-bcb2-f5e602c66244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.212109 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aecb6-63b5-4897-bcb2-f5e602c66244 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.240570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1268/2550] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:56:40 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:56:40.245213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67ca53-54cc-4535-b788-36ebe90d5c96 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.255265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67ca53-54cc-4535-b788-36ebe90d5c96 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.319258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65d4c5-f0a2-490b-8ac6-31f6e3afc66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.320017 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65d4c5-f0a2-490b-8ac6-31f6e3afc66f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.320244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65d4c5-f0a2-490b-8ac6-31f6e3afc66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.320460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65d4c5-f0a2-490b-8ac6-31f6e3afc66f admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.321181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65d4c5-f0a2-490b-8ac6-31f6e3afc66f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.323110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65d4c5-f0a2-490b-8ac6-31f6e3afc66f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.351386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1283/2551] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:56:40 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:56:40.355735 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aecb6-63b5-4897-bcb2-f5e602c66244 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.367341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aecb6-63b5-4897-bcb2-f5e602c66244 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.375974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d1b6-9acf-4145-a022-96f60dd35f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.376985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d1b6-9acf-4145-a022-96f60dd35f97 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.377347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d1b6-9acf-4145-a022-96f60dd35f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.377711 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d1b6-9acf-4145-a022-96f60dd35f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.378785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0d1b6-9acf-4145-a022-96f60dd35f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.381329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0d1b6-9acf-4145-a022-96f60dd35f97 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.393272 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1269/2552] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:40.397475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65d4c5-f0a2-490b-8ac6-31f6e3afc66f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.407982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65d4c5-f0a2-490b-8ac6-31f6e3afc66f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.413313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43445b2e-dc91-4ca9-b7cb-89ca0c7984e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.413968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43445b2e-dc91-4ca9-b7cb-89ca0c7984e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.414232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43445b2e-dc91-4ca9-b7cb-89ca0c7984e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.414508 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43445b2e-dc91-4ca9-b7cb-89ca0c7984e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.415224 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43445b2e-dc91-4ca9-b7cb-89ca0c7984e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.416883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43445b2e-dc91-4ca9-b7cb-89ca0c7984e9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.427525 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1284/2553] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:40.431895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d1b6-9acf-4145-a022-96f60dd35f97 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.442704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d1b6-9acf-4145-a022-96f60dd35f97 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.448791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e77db-6f9e-48b1-af54-466b274a9aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.449679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e77db-6f9e-48b1-af54-466b274a9aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.449959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e77db-6f9e-48b1-af54-466b274a9aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.450230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e77db-6f9e-48b1-af54-466b274a9aa2 admin admin] PATH_INFO: `/v3/projects/84f644a2c73945cbb0bda9ef43fcdeba/users/ac80b75dd90743f7ae0e72599dbd2534/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.466166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e77db-6f9e-48b1-af54-466b274a9aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f644a2c73945cbb0bda9ef43fcdeba, user_id=ac80b75dd90743f7ae0e72599dbd2534, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.468838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e77db-6f9e-48b1-af54-466b274a9aa2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.483980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1270/2554] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:56:40 2022] PUT /identity/v3/projects/84f644a2c73945cbb0bda9ef43fcdeba/users/ac80b75dd90743f7ae0e72599dbd2534/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:40.501724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a06ef1-a44c-4dd7-bd22-3c5bea8fc66f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.502074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a06ef1-a44c-4dd7-bd22-3c5bea8fc66f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.502449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a06ef1-a44c-4dd7-bd22-3c5bea8fc66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.532932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-50a06ef1-a44c-4dd7-bd22-3c5bea8fc66f None None] MFA Rules not processed for user `ac80b75dd90743f7ae0e72599dbd2534`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:40.534134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-50a06ef1-a44c-4dd7-bd22-3c5bea8fc66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:40.603807 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1285/2555] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:40.610888 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e77db-6f9e-48b1-af54-466b274a9aa2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.680069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e77db-6f9e-48b1-af54-466b274a9aa2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.686372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c78a3-7d70-4908-a5ef-e64b98d1321e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.687035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c78a3-7d70-4908-a5ef-e64b98d1321e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.687317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c78a3-7d70-4908-a5ef-e64b98d1321e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.687590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c78a3-7d70-4908-a5ef-e64b98d1321e admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.688635 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c78a3-7d70-4908-a5ef-e64b98d1321e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.690575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c78a3-7d70-4908-a5ef-e64b98d1321e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.720773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1271/2556] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:56:40 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:56:40.725617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a06ef1-a44c-4dd7-bd22-3c5bea8fc66f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.741184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a06ef1-a44c-4dd7-bd22-3c5bea8fc66f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.796684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b056a-e918-4578-8900-f3c3f65261da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.797349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b056a-e918-4578-8900-f3c3f65261da admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.797632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b056a-e918-4578-8900-f3c3f65261da admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.797914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b056a-e918-4578-8900-f3c3f65261da admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.798596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b056a-e918-4578-8900-f3c3f65261da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.800518 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b056a-e918-4578-8900-f3c3f65261da admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.825279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1286/2557] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:56:40 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:56:40.831535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c78a3-7d70-4908-a5ef-e64b98d1321e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.843618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c78a3-7d70-4908-a5ef-e64b98d1321e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.848666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b610d0-45ee-4956-be16-d18a43350e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.849322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b610d0-45ee-4956-be16-d18a43350e52 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.849601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b610d0-45ee-4956-be16-d18a43350e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.849871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b610d0-45ee-4956-be16-d18a43350e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.850587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b610d0-45ee-4956-be16-d18a43350e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.852522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b610d0-45ee-4956-be16-d18a43350e52 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.862824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1272/2558] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:40.866881 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b056a-e918-4578-8900-f3c3f65261da admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.877504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b056a-e918-4578-8900-f3c3f65261da admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.884092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43403e6a-0984-47d9-b040-53201f3aef4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.884763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43403e6a-0984-47d9-b040-53201f3aef4c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.885040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43403e6a-0984-47d9-b040-53201f3aef4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.885304 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43403e6a-0984-47d9-b040-53201f3aef4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.886026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43403e6a-0984-47d9-b040-53201f3aef4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.887584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43403e6a-0984-47d9-b040-53201f3aef4c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.899961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1287/2559] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:40.904240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b610d0-45ee-4956-be16-d18a43350e52 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:40.914643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b610d0-45ee-4956-be16-d18a43350e52 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:40.919556 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4574c14-3461-46e9-9bbf-4fe07e32f4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:40.920653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4574c14-3461-46e9-9bbf-4fe07e32f4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.920927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4574c14-3461-46e9-9bbf-4fe07e32f4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.921194 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4574c14-3461-46e9-9bbf-4fe07e32f4dd admin admin] PATH_INFO: `/v3/projects/3860a76edab04e96880c6a67bc9cb913/users/52c7eff9576d48f19ebc6de5fb5c2e46/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.937095 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4574c14-3461-46e9-9bbf-4fe07e32f4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3860a76edab04e96880c6a67bc9cb913, user_id=52c7eff9576d48f19ebc6de5fb5c2e46, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:40.940066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4574c14-3461-46e9-9bbf-4fe07e32f4dd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:40.954428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1273/2560] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:56:40 2022] PUT /identity/v3/projects/3860a76edab04e96880c6a67bc9cb913/users/52c7eff9576d48f19ebc6de5fb5c2e46/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:56:40.967941 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd99c2-a492-46c0-b400-6512fef2a3b5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:40.968333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd99c2-a492-46c0-b400-6512fef2a3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:40.968707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd99c2-a492-46c0-b400-6512fef2a3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:40.998644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a5bd99c2-a492-46c0-b400-6512fef2a3b5 None None] MFA Rules not processed for user `52c7eff9576d48f19ebc6de5fb5c2e46`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:40.999857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a5bd99c2-a492-46c0-b400-6512fef2a3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:41.068449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1288/2561] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:41.073734 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4574c14-3461-46e9-9bbf-4fe07e32f4dd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:41.140888 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4574c14-3461-46e9-9bbf-4fe07e32f4dd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:41.146585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f60f6-b930-4e5f-9db6-e2a9d522583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:41.147246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f60f6-b930-4e5f-9db6-e2a9d522583e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:41.147520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f60f6-b930-4e5f-9db6-e2a9d522583e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:41.147883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f60f6-b930-4e5f-9db6-e2a9d522583e admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:41.149003 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f60f6-b930-4e5f-9db6-e2a9d522583e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:41.150928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f60f6-b930-4e5f-9db6-e2a9d522583e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:41.180799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1274/2562] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:56:41 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:56:41.185038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd99c2-a492-46c0-b400-6512fef2a3b5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:41.196281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd99c2-a492-46c0-b400-6512fef2a3b5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:41.253977 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c4986-77df-42db-b3c4-e735a5f179ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:41.254652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c4986-77df-42db-b3c4-e735a5f179ed admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:41.254976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c4986-77df-42db-b3c4-e735a5f179ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:41.255237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c4986-77df-42db-b3c4-e735a5f179ed admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:41.255908 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c4986-77df-42db-b3c4-e735a5f179ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:41.258016 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c4986-77df-42db-b3c4-e735a5f179ed admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:41.279707 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1289/2563] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:56:41 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:56:41.283840 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f60f6-b930-4e5f-9db6-e2a9d522583e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:41.295501 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f60f6-b930-4e5f-9db6-e2a9d522583e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:41.300372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07468dea-0fbb-454b-a0ad-1c9cf7e4a00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:41.301124 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07468dea-0fbb-454b-a0ad-1c9cf7e4a00f admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:41.301403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07468dea-0fbb-454b-a0ad-1c9cf7e4a00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:41.301673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07468dea-0fbb-454b-a0ad-1c9cf7e4a00f admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:41.302395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07468dea-0fbb-454b-a0ad-1c9cf7e4a00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:41.303985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07468dea-0fbb-454b-a0ad-1c9cf7e4a00f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:41.315168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1275/2564] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:56:41.321175 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c4986-77df-42db-b3c4-e735a5f179ed admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:41.331469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c4986-77df-42db-b3c4-e735a5f179ed admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:41.337386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d30185-017a-4753-8382-c092ae209e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:41.338305 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d30185-017a-4753-8382-c092ae209e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:41.338581 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d30185-017a-4753-8382-c092ae209e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:41.338849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d30185-017a-4753-8382-c092ae209e59 admin admin] PATH_INFO: `/v3/projects/51dc03f092d84e64870e24a8f7e4cb8d/users/7143016d19af4f26a4f07cd7cd852131/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:41.354404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d30185-017a-4753-8382-c092ae209e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51dc03f092d84e64870e24a8f7e4cb8d, user_id=7143016d19af4f26a4f07cd7cd852131, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:41.357179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d30185-017a-4753-8382-c092ae209e59 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:41.372412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1290/2565] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:56:41 2022] PUT /identity/v3/projects/51dc03f092d84e64870e24a8f7e4cb8d/users/7143016d19af4f26a4f07cd7cd852131/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:56:41.378779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07468dea-0fbb-454b-a0ad-1c9cf7e4a00f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:41.390458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07468dea-0fbb-454b-a0ad-1c9cf7e4a00f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:41.448457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963e354-e92e-4edd-a1e7-022e2b7210b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:41.449067 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963e354-e92e-4edd-a1e7-022e2b7210b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:41.449342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963e354-e92e-4edd-a1e7-022e2b7210b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:41.449579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963e354-e92e-4edd-a1e7-022e2b7210b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:41.450327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963e354-e92e-4edd-a1e7-022e2b7210b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:41.451924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963e354-e92e-4edd-a1e7-022e2b7210b6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:41.461971 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1276/2566] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:41.466390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d30185-017a-4753-8382-c092ae209e59 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:41.477788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d30185-017a-4753-8382-c092ae209e59 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:41.483877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4332911-80ed-40bb-b751-39210abd021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:41.484793 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4332911-80ed-40bb-b751-39210abd021a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:41.485065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4332911-80ed-40bb-b751-39210abd021a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:41.485333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4332911-80ed-40bb-b751-39210abd021a admin admin] PATH_INFO: `/v3/projects/51dc03f092d84e64870e24a8f7e4cb8d/users/7143016d19af4f26a4f07cd7cd852131/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:41.487567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4332911-80ed-40bb-b751-39210abd021a admin admin] RBAC: Authorizing `identity:create_grant(project_id=51dc03f092d84e64870e24a8f7e4cb8d, user_id=7143016d19af4f26a4f07cd7cd852131, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:41.490209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4332911-80ed-40bb-b751-39210abd021a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:41.501206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1291/2567] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:56:41 2022] PUT /identity/v3/projects/51dc03f092d84e64870e24a8f7e4cb8d/users/7143016d19af4f26a4f07cd7cd852131/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:41.515854 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000fbf-133c-4769-a743-f82f6ccc0e80 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:41.516415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000fbf-133c-4769-a743-f82f6ccc0e80 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:41.516415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000fbf-133c-4769-a743-f82f6ccc0e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:41.546674 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-34000fbf-133c-4769-a743-f82f6ccc0e80 None None] MFA Rules not processed for user `7143016d19af4f26a4f07cd7cd852131`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:41.547894 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-34000fbf-133c-4769-a743-f82f6ccc0e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:41.621491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1277/2568] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:41.633835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4332911-80ed-40bb-b751-39210abd021a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:41.635726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f4332911-80ed-40bb-b751-39210abd021a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:41.693438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4332911-80ed-40bb-b751-39210abd021a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:41.699417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99f854-6d6c-4e70-9efc-16934cadac9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:41.700001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99f854-6d6c-4e70-9efc-16934cadac9f service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:41.700278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99f854-6d6c-4e70-9efc-16934cadac9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:41.700571 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99f854-6d6c-4e70-9efc-16934cadac9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:41.760673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99f854-6d6c-4e70-9efc-16934cadac9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:41.762122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99f854-6d6c-4e70-9efc-16934cadac9f service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:41.771754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1292/2569] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:43.338857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34000fbf-133c-4769-a743-f82f6ccc0e80 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:43.340518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-34000fbf-133c-4769-a743-f82f6ccc0e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:43.353302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34000fbf-133c-4769-a743-f82f6ccc0e80 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:43.357311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c33f86-854b-470a-b13b-48f9575c48ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:43.357957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c33f86-854b-470a-b13b-48f9575c48ce service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:43.358235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c33f86-854b-470a-b13b-48f9575c48ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:43.358505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c33f86-854b-470a-b13b-48f9575c48ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:43.419473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c33f86-854b-470a-b13b-48f9575c48ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:43.420863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c33f86-854b-470a-b13b-48f9575c48ce service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:43.430045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1278/2570] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:44.028882 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99f854-6d6c-4e70-9efc-16934cadac9f service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:44.039767 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99f854-6d6c-4e70-9efc-16934cadac9f service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:44.044943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94deb487-a2ad-4d6b-8977-6155869f8a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:44.045587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deb487-a2ad-4d6b-8977-6155869f8a7b service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:44.045919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deb487-a2ad-4d6b-8977-6155869f8a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:44.046188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deb487-a2ad-4d6b-8977-6155869f8a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:44.056674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94deb487-a2ad-4d6b-8977-6155869f8a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:44.058461 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94deb487-a2ad-4d6b-8977-6155869f8a7b service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:44.068894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1293/2571] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:48.000203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c33f86-854b-470a-b13b-48f9575c48ce service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:48.080077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c33f86-854b-470a-b13b-48f9575c48ce service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:48.086296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:48.087170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:48.087511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:48.087685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] PATH_INFO: `/v3/users/ac80b75dd90743f7ae0e72599dbd2534` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:48.089491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac80b75dd90743f7ae0e72599dbd2534)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:48.091443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:48.143213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac80b75dd90743f7ae0e72599dbd2534'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.143824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac80b75dd90743f7ae0e72599dbd2534'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.155907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac80b75dd90743f7ae0e72599dbd2534'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.162372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac80b75dd90743f7ae0e72599dbd2534'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.170769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac80b75dd90743f7ae0e72599dbd2534'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.176696 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1279/2572] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:56:47 2022] DELETE /identity/v3/users/ac80b75dd90743f7ae0e72599dbd2534 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:48.181201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acb69b-4b4f-4e40-97f1-e9e627e05fac None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:48.181694 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acb69b-4b4f-4e40-97f1-e9e627e05fac None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:48.181794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acb69b-4b4f-4e40-97f1-e9e627e05fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:48.199491 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-48acb69b-4b4f-4e40-97f1-e9e627e05fac None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:48.200832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-48acb69b-4b4f-4e40-97f1-e9e627e05fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:48.260660 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1294/2573] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:48.272075 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:48.274434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:48.338140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5a6ae-a4cf-474e-9032-f48892cac0c8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:48.342563 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e25fd-dcde-4728-9e54-9f901bc8026e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:48.343206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e25fd-dcde-4728-9e54-9f901bc8026e service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:48.343492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e25fd-dcde-4728-9e54-9f901bc8026e service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:48.343733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e25fd-dcde-4728-9e54-9f901bc8026e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:48.352884 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e25fd-dcde-4728-9e54-9f901bc8026e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:48.354317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e25fd-dcde-4728-9e54-9f901bc8026e service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:48.363708 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1280/2574] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:48.626729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acb69b-4b4f-4e40-97f1-e9e627e05fac None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:48.628230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-48acb69b-4b4f-4e40-97f1-e9e627e05fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:48.640146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acb69b-4b4f-4e40-97f1-e9e627e05fac None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:48.645499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:48.646351 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:48.646636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:48.646900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] PATH_INFO: `/v3/projects/84f644a2c73945cbb0bda9ef43fcdeba` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:48.648089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f644a2c73945cbb0bda9ef43fcdeba)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:48.650018 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:48.697241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f644a2c73945cbb0bda9ef43fcdeba', 'request_id': 'req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.706537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f644a2c73945cbb0bda9ef43fcdeba', 'request_id': 'req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.712693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f644a2c73945cbb0bda9ef43fcdeba', 'request_id': 'req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.724809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] The token cache is being invalidate because project 84f644a2c73945cbb0bda9ef43fcdeba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:56:48.725094 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.725980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.727763 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1295/2575] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:56:48 2022] DELETE /identity/v3/projects/84f644a2c73945cbb0bda9ef43fcdeba => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:48.731930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e25fd-dcde-4728-9e54-9f901bc8026e service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:48.741832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e25fd-dcde-4728-9e54-9f901bc8026e service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:48.802112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:48.803069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:48.803410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:48.803686 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] PATH_INFO: `/v3/users/52c7eff9576d48f19ebc6de5fb5c2e46` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:48.804939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c7eff9576d48f19ebc6de5fb5c2e46)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:48.806898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:48.853978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c7eff9576d48f19ebc6de5fb5c2e46'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.854617 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c7eff9576d48f19ebc6de5fb5c2e46'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.867825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c7eff9576d48f19ebc6de5fb5c2e46'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.875587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c7eff9576d48f19ebc6de5fb5c2e46'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.886919 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c7eff9576d48f19ebc6de5fb5c2e46'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:48.895899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1281/2576] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:56:48 2022] DELETE /identity/v3/users/52c7eff9576d48f19ebc6de5fb5c2e46 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:49.160136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:49.161731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:49.176517 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f39baf-b022-4d4c-aa9e-a90253a67d2a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:49.233613 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:49.234479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:49.234761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:49.235030 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] PATH_INFO: `/v3/projects/3860a76edab04e96880c6a67bc9cb913` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:49.236212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3860a76edab04e96880c6a67bc9cb913)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:49.238201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:49.288568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3860a76edab04e96880c6a67bc9cb913', 'request_id': 'req-4a09b282-874c-4ca1-9eb1-14a0c39418c8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.297466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3860a76edab04e96880c6a67bc9cb913', 'request_id': 'req-4a09b282-874c-4ca1-9eb1-14a0c39418c8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.303919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3860a76edab04e96880c6a67bc9cb913', 'request_id': 'req-4a09b282-874c-4ca1-9eb1-14a0c39418c8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.314433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] The token cache is being invalidate because project 3860a76edab04e96880c6a67bc9cb913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:56:49.314710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.315344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.316899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1296/2577] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:56:49 2022] DELETE /identity/v3/projects/3860a76edab04e96880c6a67bc9cb913 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:49.321093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:49.322581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:49.335578 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958e9ed-a131-4a73-a13f-47193e1496a0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:49.398942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:49.400245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:49.400540 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:49.400966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] PATH_INFO: `/v3/users/7143016d19af4f26a4f07cd7cd852131` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:49.403022 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7143016d19af4f26a4f07cd7cd852131)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:49.405942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:49.457212 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7143016d19af4f26a4f07cd7cd852131'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.457868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7143016d19af4f26a4f07cd7cd852131'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.469404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7143016d19af4f26a4f07cd7cd852131'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.476841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7143016d19af4f26a4f07cd7cd852131'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.487571 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b5b6010-0a6e-4304-9659-034325887c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7143016d19af4f26a4f07cd7cd852131'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.494302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1282/2578] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:56:49 2022] DELETE /identity/v3/users/7143016d19af4f26a4f07cd7cd852131 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:49.781438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:49.782980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:49.796323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09b282-874c-4ca1-9eb1-14a0c39418c8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:49.860668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:49.861561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:49.861835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:49.862099 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] PATH_INFO: `/v3/projects/51dc03f092d84e64870e24a8f7e4cb8d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:49.863446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] RBAC: Authorizing `identity:delete_project(project_id=51dc03f092d84e64870e24a8f7e4cb8d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:49.865398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:49.913664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51dc03f092d84e64870e24a8f7e4cb8d', 'request_id': 'req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.923663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51dc03f092d84e64870e24a8f7e4cb8d', 'request_id': 'req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.930041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51dc03f092d84e64870e24a8f7e4cb8d', 'request_id': 'req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.940639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] The token cache is being invalidate because project 51dc03f092d84e64870e24a8f7e4cb8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:56:49.940929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.941588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-85120ac6-9dda-4ddf-a9d1-f5f4bf1d321e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:56:49.943144 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1297/2579] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:56:49 2022] DELETE /identity/v3/projects/51dc03f092d84e64870e24a8f7e4cb8d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:49.948397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f7e12-a3b6-4d0a-ae9e-2798bd08887e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:49.949046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f7e12-a3b6-4d0a-ae9e-2798bd08887e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:49.949046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f7e12-a3b6-4d0a-ae9e-2798bd08887e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:49.967514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9e9f7e12-a3b6-4d0a-ae9e-2798bd08887e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:49.968738 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9e9f7e12-a3b6-4d0a-ae9e-2798bd08887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:50.035845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1283/2580] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:50.043878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59acbf-7e76-48b1-8fa5-6f1361eb3b6b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:50.044166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59acbf-7e76-48b1-8fa5-6f1361eb3b6b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:50.044443 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59acbf-7e76-48b1-8fa5-6f1361eb3b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:50.066171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2e59acbf-7e76-48b1-8fa5-6f1361eb3b6b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:50.067997 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2e59acbf-7e76-48b1-8fa5-6f1361eb3b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:50.079145 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1298/2581] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:50.083690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f7e12-a3b6-4d0a-ae9e-2798bd08887e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:50.093752 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f7e12-a3b6-4d0a-ae9e-2798bd08887e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:50.099416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63f9a6-6c4f-4ba4-a6a3-f4c3bdcfff34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:50.100068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63f9a6-6c4f-4ba4-a6a3-f4c3bdcfff34 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:50.100343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63f9a6-6c4f-4ba4-a6a3-f4c3bdcfff34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:50.100643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63f9a6-6c4f-4ba4-a6a3-f4c3bdcfff34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:50.101359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63f9a6-6c4f-4ba4-a6a3-f4c3bdcfff34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:56:50.101660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63f9a6-6c4f-4ba4-a6a3-f4c3bdcfff34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:50.103257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63f9a6-6c4f-4ba4-a6a3-f4c3bdcfff34 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:50.114985 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1284/2582] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:56:50.119742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59acbf-7e76-48b1-8fa5-6f1361eb3b6b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:50.137712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59acbf-7e76-48b1-8fa5-6f1361eb3b6b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:50.144633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad8a73-fb49-4f9c-9036-0986baa21604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:50.145299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad8a73-fb49-4f9c-9036-0986baa21604 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:50.145578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad8a73-fb49-4f9c-9036-0986baa21604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:50.145848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad8a73-fb49-4f9c-9036-0986baa21604 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:50.146528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad8a73-fb49-4f9c-9036-0986baa21604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:50.148437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad8a73-fb49-4f9c-9036-0986baa21604 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:50.183097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1299/2583] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:56:50 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:56:50.190152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63f9a6-6c4f-4ba4-a6a3-f4c3bdcfff34 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:50.200946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63f9a6-6c4f-4ba4-a6a3-f4c3bdcfff34 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:50.262614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d59b8-78b7-4369-a793-3a8626348aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:50.263280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d59b8-78b7-4369-a793-3a8626348aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:50.263542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d59b8-78b7-4369-a793-3a8626348aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:50.263803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d59b8-78b7-4369-a793-3a8626348aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:50.264481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d59b8-78b7-4369-a793-3a8626348aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:50.266443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d59b8-78b7-4369-a793-3a8626348aeb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:50.289220 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1285/2584] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:56:50 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:56:50.295328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad8a73-fb49-4f9c-9036-0986baa21604 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:50.306015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad8a73-fb49-4f9c-9036-0986baa21604 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:50.311440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6838f-69e0-4783-979d-03a80c9f5042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:50.312176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6838f-69e0-4783-979d-03a80c9f5042 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:50.312568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6838f-69e0-4783-979d-03a80c9f5042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:50.312951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6838f-69e0-4783-979d-03a80c9f5042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:50.314448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6838f-69e0-4783-979d-03a80c9f5042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:50.316968 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6838f-69e0-4783-979d-03a80c9f5042 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:50.330115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1300/2585] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:50.334323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d59b8-78b7-4369-a793-3a8626348aeb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:50.345130 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d59b8-78b7-4369-a793-3a8626348aeb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:50.350213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623103a9-0d22-4d99-aee7-80c180d3fc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:50.350870 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623103a9-0d22-4d99-aee7-80c180d3fc9e admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:50.351153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623103a9-0d22-4d99-aee7-80c180d3fc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:50.351426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623103a9-0d22-4d99-aee7-80c180d3fc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:50.352151 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623103a9-0d22-4d99-aee7-80c180d3fc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:50.353774 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623103a9-0d22-4d99-aee7-80c180d3fc9e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:50.363920 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1286/2586] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:56:50.368040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6838f-69e0-4783-979d-03a80c9f5042 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:50.379007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6838f-69e0-4783-979d-03a80c9f5042 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:50.385341 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3c1d-dbaa-42f6-beff-69fd74046736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:50.386266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3c1d-dbaa-42f6-beff-69fd74046736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:50.386543 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3c1d-dbaa-42f6-beff-69fd74046736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:50.386828 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3c1d-dbaa-42f6-beff-69fd74046736 admin admin] PATH_INFO: `/v3/projects/004ea4b077a549868292dad9caaa5515/users/9558de3a3cd84b8fa24fdbe553ca706a/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:50.402450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f3c1d-dbaa-42f6-beff-69fd74046736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=004ea4b077a549868292dad9caaa5515, user_id=9558de3a3cd84b8fa24fdbe553ca706a, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:50.405200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f3c1d-dbaa-42f6-beff-69fd74046736 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:50.419938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1301/2587] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:56:50 2022] PUT /identity/v3/projects/004ea4b077a549868292dad9caaa5515/users/9558de3a3cd84b8fa24fdbe553ca706a/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:56:50.428057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65232cff-3f20-4e1b-a54c-346d994d8eea None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:50.428472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65232cff-3f20-4e1b-a54c-346d994d8eea None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:50.428959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65232cff-3f20-4e1b-a54c-346d994d8eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:50.450660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-65232cff-3f20-4e1b-a54c-346d994d8eea None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:50.451847 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-65232cff-3f20-4e1b-a54c-346d994d8eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:50.519431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1287/2588] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:50.529510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3c1d-dbaa-42f6-beff-69fd74046736 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:50.531533 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d24f3c1d-dbaa-42f6-beff-69fd74046736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:50.592536 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3c1d-dbaa-42f6-beff-69fd74046736 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:50.596481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f22c5-7c66-4e56-978b-5aa9b4f2306d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:50.597149 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f22c5-7c66-4e56-978b-5aa9b4f2306d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:50.597402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f22c5-7c66-4e56-978b-5aa9b4f2306d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:50.597691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f22c5-7c66-4e56-978b-5aa9b4f2306d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:50.608140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f22c5-7c66-4e56-978b-5aa9b4f2306d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:50.609637 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f22c5-7c66-4e56-978b-5aa9b4f2306d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:50.619553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1302/2589] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:51.238735 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9337f-1730-443d-8c85-8b7d5108bfab None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:51.239018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9337f-1730-443d-8c85-8b7d5108bfab None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:51.239290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9337f-1730-443d-8c85-8b7d5108bfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:51.267995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1fd9337f-1730-443d-8c85-8b7d5108bfab None None] MFA Rules not processed for user `9558de3a3cd84b8fa24fdbe553ca706a`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:51.269284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1fd9337f-1730-443d-8c85-8b7d5108bfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:51.337771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1288/2590] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:51.348909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f22c5-7c66-4e56-978b-5aa9b4f2306d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:51.350330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-448f22c5-7c66-4e56-978b-5aa9b4f2306d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:51.406731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f22c5-7c66-4e56-978b-5aa9b4f2306d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:51.410716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c6153-1731-4277-bd65-eae4ea04a37d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:51.411365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c6153-1731-4277-bd65-eae4ea04a37d service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:51.411636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c6153-1731-4277-bd65-eae4ea04a37d service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:51.411909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c6153-1731-4277-bd65-eae4ea04a37d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:51.421051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c6153-1731-4277-bd65-eae4ea04a37d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:51.422496 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c6153-1731-4277-bd65-eae4ea04a37d service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:51.431237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1303/2591] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:53.853631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d30d1-5b04-4c72-93c5-1e4011a47ddc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:53.854261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d30d1-5b04-4c72-93c5-1e4011a47ddc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:53.854261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d30d1-5b04-4c72-93c5-1e4011a47ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:53.873100 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d05d30d1-5b04-4c72-93c5-1e4011a47ddc None None] MFA Rules not processed for user `9558de3a3cd84b8fa24fdbe553ca706a`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:56:53.874336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d05d30d1-5b04-4c72-93c5-1e4011a47ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:53.885748 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1289/2592] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:56:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:56:54.121808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c6153-1731-4277-bd65-eae4ea04a37d service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:54.123366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-037c6153-1731-4277-bd65-eae4ea04a37d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:54.136157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c6153-1731-4277-bd65-eae4ea04a37d service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:54.140373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cc5be-c519-48a9-b3dc-bfabd57070e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:54.141051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cc5be-c519-48a9-b3dc-bfabd57070e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:54.141337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cc5be-c519-48a9-b3dc-bfabd57070e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:54.141617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cc5be-c519-48a9-b3dc-bfabd57070e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:54.143473 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f62cc5be-c519-48a9-b3dc-bfabd57070e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:54.154486 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cc5be-c519-48a9-b3dc-bfabd57070e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:54.155919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cc5be-c519-48a9-b3dc-bfabd57070e7 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:54.164290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1304/2593] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:56:55.416878 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d30d1-5b04-4c72-93c5-1e4011a47ddc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:56:55.418689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d05d30d1-5b04-4c72-93c5-1e4011a47ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:55.510450 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d30d1-5b04-4c72-93c5-1e4011a47ddc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:56:55.517682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb4d22-c221-4a3d-9019-34588a9044d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e934c8e7dd0425cac68eb91e5e2c0ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:56:55.518580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb4d22-c221-4a3d-9019-34588a9044d7 service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:56:55.518887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb4d22-c221-4a3d-9019-34588a9044d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:56:55.519231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb4d22-c221-4a3d-9019-34588a9044d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:56:55.522516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9aeb4d22-c221-4a3d-9019-34588a9044d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:56:55.540934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb4d22-c221-4a3d-9019-34588a9044d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:56:55.543532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb4d22-c221-4a3d-9019-34588a9044d7 service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:56:55.554182 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1290/2594] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:56:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:02.862965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cc5be-c519-48a9-b3dc-bfabd57070e7 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:02.872199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cc5be-c519-48a9-b3dc-bfabd57070e7 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:02.876079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f68982-28a9-447d-8ee9-7adba7228cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:02.876840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f68982-28a9-447d-8ee9-7adba7228cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:02.877192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f68982-28a9-447d-8ee9-7adba7228cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:02.877530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f68982-28a9-447d-8ee9-7adba7228cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:02.879507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-39f68982-28a9-447d-8ee9-7adba7228cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:02.892670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f68982-28a9-447d-8ee9-7adba7228cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:02.894225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f68982-28a9-447d-8ee9-7adba7228cc6 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:02.905092 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1305/2595] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:06.386694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb4d22-c221-4a3d-9019-34588a9044d7 service placement] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:06.388581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9aeb4d22-c221-4a3d-9019-34588a9044d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:06.402417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb4d22-c221-4a3d-9019-34588a9044d7 service placement] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:06.407960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:06.411073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:06.411073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:06.411073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] PATH_INFO: `/v3/users/49aaea67fba64d2583d2023505ce7779` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:06.411073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=49aaea67fba64d2583d2023505ce7779)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:06.413056 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:06.469406 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49aaea67fba64d2583d2023505ce7779'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:06.470054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49aaea67fba64d2583d2023505ce7779'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:06.482361 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49aaea67fba64d2583d2023505ce7779'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:06.489308 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49aaea67fba64d2583d2023505ce7779'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:06.498422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4c9c970e-e635-4458-bef2-b85511b09f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49aaea67fba64d2583d2023505ce7779'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:06.505603 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1291/2596] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:57:06 2022] DELETE /identity/v3/users/49aaea67fba64d2583d2023505ce7779 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:06.755430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f68982-28a9-447d-8ee9-7adba7228cc6 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:06.765101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f68982-28a9-447d-8ee9-7adba7228cc6 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:06.830363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:06.831241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:06.831528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:06.831801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] PATH_INFO: `/v3/projects/e9f31cfe6a484184b4b4873bb918b266` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:06.833108 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f31cfe6a484184b4b4873bb918b266)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:06.835079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:06.888347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f31cfe6a484184b4b4873bb918b266', 'request_id': 'req-f6371053-a12e-4238-8aa1-b3bbb5219342'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:06.898020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f31cfe6a484184b4b4873bb918b266', 'request_id': 'req-f6371053-a12e-4238-8aa1-b3bbb5219342'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:06.904760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f31cfe6a484184b4b4873bb918b266', 'request_id': 'req-f6371053-a12e-4238-8aa1-b3bbb5219342'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:06.915502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] The token cache is being invalidate because project e9f31cfe6a484184b4b4873bb918b266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:57:06.915788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:06.916516 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f6371053-a12e-4238-8aa1-b3bbb5219342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:06.918176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1306/2597] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:57:06 2022] DELETE /identity/v3/projects/e9f31cfe6a484184b4b4873bb918b266 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:06.923605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d883d9-a46d-4bd9-9298-452e31ff9fd8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:06.923911 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d883d9-a46d-4bd9-9298-452e31ff9fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:06.924148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d883d9-a46d-4bd9-9298-452e31ff9fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:06.946065 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c0d883d9-a46d-4bd9-9298-452e31ff9fd8 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:06.947257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c0d883d9-a46d-4bd9-9298-452e31ff9fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:07.006329 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1292/2598] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:07.014481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c91a9-4694-4487-8a75-92aa94ef5633 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.014770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c91a9-4694-4487-8a75-92aa94ef5633 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.015050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c91a9-4694-4487-8a75-92aa94ef5633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.033305 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b96c91a9-4694-4487-8a75-92aa94ef5633 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:07.034381 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b96c91a9-4694-4487-8a75-92aa94ef5633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:07.042528 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1307/2599] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:57:07.047027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d883d9-a46d-4bd9-9298-452e31ff9fd8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.057221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d883d9-a46d-4bd9-9298-452e31ff9fd8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.062608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74d460-789d-4397-9565-206fb6e613ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.063631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74d460-789d-4397-9565-206fb6e613ec admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.064004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74d460-789d-4397-9565-206fb6e613ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.064279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74d460-789d-4397-9565-206fb6e613ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.065036 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74d460-789d-4397-9565-206fb6e613ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:57:07.065342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74d460-789d-4397-9565-206fb6e613ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.066942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74d460-789d-4397-9565-206fb6e613ec admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.078341 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1293/2600] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:57:07.083025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c91a9-4694-4487-8a75-92aa94ef5633 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.097540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c91a9-4694-4487-8a75-92aa94ef5633 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.106040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948cc7f-63f4-443b-8ed7-5063d2f3b84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.107101 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948cc7f-63f4-443b-8ed7-5063d2f3b84e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.107269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948cc7f-63f4-443b-8ed7-5063d2f3b84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.107553 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948cc7f-63f4-443b-8ed7-5063d2f3b84e admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.108281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948cc7f-63f4-443b-8ed7-5063d2f3b84e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.110199 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948cc7f-63f4-443b-8ed7-5063d2f3b84e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.139616 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1308/2601] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:07 2022] POST /identity/v3/projects => generated 357 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:07.144004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74d460-789d-4397-9565-206fb6e613ec admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.154523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74d460-789d-4397-9565-206fb6e613ec admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.224418 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623c812-6799-4d8a-ad87-b62e5f9119da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.225169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623c812-6799-4d8a-ad87-b62e5f9119da admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.225511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623c812-6799-4d8a-ad87-b62e5f9119da admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.225836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623c812-6799-4d8a-ad87-b62e5f9119da admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.226687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623c812-6799-4d8a-ad87-b62e5f9119da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.229648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623c812-6799-4d8a-ad87-b62e5f9119da admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.256428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1294/2602] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:07 2022] POST /identity/v3/users => generated 279 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:07.260692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948cc7f-63f4-443b-8ed7-5063d2f3b84e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.271395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948cc7f-63f4-443b-8ed7-5063d2f3b84e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.276666 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ace7e-b3a2-4a88-a88f-ac7c8518a464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.277334 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ace7e-b3a2-4a88-a88f-ac7c8518a464 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.277618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ace7e-b3a2-4a88-a88f-ac7c8518a464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.277873 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ace7e-b3a2-4a88-a88f-ac7c8518a464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.278625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ace7e-b3a2-4a88-a88f-ac7c8518a464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.280269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ace7e-b3a2-4a88-a88f-ac7c8518a464 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.290741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1309/2603] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:07.294887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623c812-6799-4d8a-ad87-b62e5f9119da admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.305475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623c812-6799-4d8a-ad87-b62e5f9119da admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.310870 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3dd52-794d-4aeb-b86f-fc8b22f9e375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.311529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3dd52-794d-4aeb-b86f-fc8b22f9e375 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.311814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3dd52-794d-4aeb-b86f-fc8b22f9e375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.312173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3dd52-794d-4aeb-b86f-fc8b22f9e375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.313300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3dd52-794d-4aeb-b86f-fc8b22f9e375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.314907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3dd52-794d-4aeb-b86f-fc8b22f9e375 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.324910 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1295/2604] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:07.329056 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ace7e-b3a2-4a88-a88f-ac7c8518a464 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.338925 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ace7e-b3a2-4a88-a88f-ac7c8518a464 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.344215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec95b1-f602-447a-981b-21753f76e844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.345137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec95b1-f602-447a-981b-21753f76e844 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.345417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec95b1-f602-447a-981b-21753f76e844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.345678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec95b1-f602-447a-981b-21753f76e844 admin admin] PATH_INFO: `/v3/projects/6876653af37247e8a08852f3ad15bd45/users/da75c29e5faf47409d90ef22108f5890/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.361182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec95b1-f602-447a-981b-21753f76e844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6876653af37247e8a08852f3ad15bd45, user_id=da75c29e5faf47409d90ef22108f5890, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.363746 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec95b1-f602-447a-981b-21753f76e844 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.378227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1310/2605] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:07 2022] PUT /identity/v3/projects/6876653af37247e8a08852f3ad15bd45/users/da75c29e5faf47409d90ef22108f5890/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:07.388557 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afa7c6-4d69-43bb-9d93-e44902e940be None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.388557 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afa7c6-4d69-43bb-9d93-e44902e940be None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.389638 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afa7c6-4d69-43bb-9d93-e44902e940be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.416322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-45afa7c6-4d69-43bb-9d93-e44902e940be None None] MFA Rules not processed for user `da75c29e5faf47409d90ef22108f5890`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:07.417577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-45afa7c6-4d69-43bb-9d93-e44902e940be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:07.485438 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1296/2606] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:07 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:07.490097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec95b1-f602-447a-981b-21753f76e844 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.555356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec95b1-f602-447a-981b-21753f76e844 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.561150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bc7ec-7e85-421f-9b04-485fcff49ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.561810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bc7ec-7e85-421f-9b04-485fcff49ace admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.562087 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bc7ec-7e85-421f-9b04-485fcff49ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.562356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bc7ec-7e85-421f-9b04-485fcff49ace admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.563407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41bc7ec-7e85-421f-9b04-485fcff49ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.565329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41bc7ec-7e85-421f-9b04-485fcff49ace admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.595242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1311/2607] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:07 2022] POST /identity/v3/projects => generated 355 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:07.599828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afa7c6-4d69-43bb-9d93-e44902e940be None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.613321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afa7c6-4d69-43bb-9d93-e44902e940be None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.675648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89f1f9-f5c3-4560-a4e7-09d95060d772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.676323 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89f1f9-f5c3-4560-a4e7-09d95060d772 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.676628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89f1f9-f5c3-4560-a4e7-09d95060d772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.676872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89f1f9-f5c3-4560-a4e7-09d95060d772 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.677569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89f1f9-f5c3-4560-a4e7-09d95060d772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.679722 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89f1f9-f5c3-4560-a4e7-09d95060d772 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.701857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1297/2608] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:07 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:07.705760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bc7ec-7e85-421f-9b04-485fcff49ace admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.716001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bc7ec-7e85-421f-9b04-485fcff49ace admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.720789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bd5d2-2872-45e3-98ee-e0785bc9d65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.721439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bd5d2-2872-45e3-98ee-e0785bc9d65e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.721718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bd5d2-2872-45e3-98ee-e0785bc9d65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.721988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bd5d2-2872-45e3-98ee-e0785bc9d65e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.722761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bd5d2-2872-45e3-98ee-e0785bc9d65e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.724332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bd5d2-2872-45e3-98ee-e0785bc9d65e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.735718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1312/2609] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:07.739978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89f1f9-f5c3-4560-a4e7-09d95060d772 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.749952 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89f1f9-f5c3-4560-a4e7-09d95060d772 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.754938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a46ee0-b31c-4cb5-8147-fbe65c759e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.755820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a46ee0-b31c-4cb5-8147-fbe65c759e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.756099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a46ee0-b31c-4cb5-8147-fbe65c759e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.756363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a46ee0-b31c-4cb5-8147-fbe65c759e41 admin admin] PATH_INFO: `/v3/projects/ccf1d3e8fa5947ce869f4f90174e37e4/users/cd45a0a0c7cc440888ef5541db5fa373/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.771591 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a46ee0-b31c-4cb5-8147-fbe65c759e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf1d3e8fa5947ce869f4f90174e37e4, user_id=cd45a0a0c7cc440888ef5541db5fa373, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.773877 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a46ee0-b31c-4cb5-8147-fbe65c759e41 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.788106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1298/2610] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:07 2022] PUT /identity/v3/projects/ccf1d3e8fa5947ce869f4f90174e37e4/users/cd45a0a0c7cc440888ef5541db5fa373/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:57:07.792027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bd5d2-2872-45e3-98ee-e0785bc9d65e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.802045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bd5d2-2872-45e3-98ee-e0785bc9d65e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.859219 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee11412-2383-4f73-a1b9-b31db3b57e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.859890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11412-2383-4f73-a1b9-b31db3b57e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.860137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11412-2383-4f73-a1b9-b31db3b57e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.860410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11412-2383-4f73-a1b9-b31db3b57e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.861171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee11412-2383-4f73-a1b9-b31db3b57e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.862760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee11412-2383-4f73-a1b9-b31db3b57e36 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.872297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1313/2611] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:07.876588 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a46ee0-b31c-4cb5-8147-fbe65c759e41 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:07.887181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a46ee0-b31c-4cb5-8147-fbe65c759e41 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:07.892140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82487aef-0a8b-45f5-a5f9-42cdb3192789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:07.893088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82487aef-0a8b-45f5-a5f9-42cdb3192789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.893387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82487aef-0a8b-45f5-a5f9-42cdb3192789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.893656 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82487aef-0a8b-45f5-a5f9-42cdb3192789 admin admin] PATH_INFO: `/v3/projects/ccf1d3e8fa5947ce869f4f90174e37e4/users/cd45a0a0c7cc440888ef5541db5fa373/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.895639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82487aef-0a8b-45f5-a5f9-42cdb3192789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf1d3e8fa5947ce869f4f90174e37e4, user_id=cd45a0a0c7cc440888ef5541db5fa373, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:07.898311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82487aef-0a8b-45f5-a5f9-42cdb3192789 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:07.908892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1299/2612] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:07 2022] PUT /identity/v3/projects/ccf1d3e8fa5947ce869f4f90174e37e4/users/cd45a0a0c7cc440888ef5541db5fa373/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:07.920736 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbf0e9-7ab2-484d-8ea1-97ee85508a6e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:07.921029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbf0e9-7ab2-484d-8ea1-97ee85508a6e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:07.921308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbf0e9-7ab2-484d-8ea1-97ee85508a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:07.950072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f8bbf0e9-7ab2-484d-8ea1-97ee85508a6e None None] MFA Rules not processed for user `cd45a0a0c7cc440888ef5541db5fa373`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:07.951740 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f8bbf0e9-7ab2-484d-8ea1-97ee85508a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:08.025979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1314/2613] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:07 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:08.041074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82487aef-0a8b-45f5-a5f9-42cdb3192789 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:08.043123 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-82487aef-0a8b-45f5-a5f9-42cdb3192789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:08.108616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82487aef-0a8b-45f5-a5f9-42cdb3192789 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:08.112791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673d322-aa99-4672-a14e-e5bdf0d7a27b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:08.113429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d322-aa99-4672-a14e-e5bdf0d7a27b service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:08.113703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d322-aa99-4672-a14e-e5bdf0d7a27b service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:08.113972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d322-aa99-4672-a14e-e5bdf0d7a27b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:08.123528 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673d322-aa99-4672-a14e-e5bdf0d7a27b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:08.124929 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673d322-aa99-4672-a14e-e5bdf0d7a27b service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:08.134156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1300/2614] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:08 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:08.212805 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbf0e9-7ab2-484d-8ea1-97ee85508a6e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:08.223046 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbf0e9-7ab2-484d-8ea1-97ee85508a6e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:08.227065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299870ea-523d-4601-89d4-d0b3df5b2b53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:08.227718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299870ea-523d-4601-89d4-d0b3df5b2b53 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:08.227999 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299870ea-523d-4601-89d4-d0b3df5b2b53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:08.228270 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299870ea-523d-4601-89d4-d0b3df5b2b53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:08.286445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299870ea-523d-4601-89d4-d0b3df5b2b53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:08.287876 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299870ea-523d-4601-89d4-d0b3df5b2b53 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:08.296909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1315/2615] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:08 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:15.706353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b43f17-28cd-4c1b-873c-abf8bae4f27c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:15.707137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b43f17-28cd-4c1b-873c-abf8bae4f27c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:15.707302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b43f17-28cd-4c1b-873c-abf8bae4f27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:15.729343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-87b43f17-28cd-4c1b-873c-abf8bae4f27c None None] MFA Rules not processed for user `9558de3a3cd84b8fa24fdbe553ca706a`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:15.730518 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-87b43f17-28cd-4c1b-873c-abf8bae4f27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:15.789336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1301/2616] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:25.344916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299870ea-523d-4601-89d4-d0b3df5b2b53 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:25.409175 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299870ea-523d-4601-89d4-d0b3df5b2b53 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:25.414222 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:25.415099 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:25.415373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:25.415636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] PATH_INFO: `/v3/users/da75c29e5faf47409d90ef22108f5890` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:25.416851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da75c29e5faf47409d90ef22108f5890)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:25.418745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:25.470332 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da75c29e5faf47409d90ef22108f5890'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:25.471013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da75c29e5faf47409d90ef22108f5890'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:25.482822 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da75c29e5faf47409d90ef22108f5890'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:25.491223 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da75c29e5faf47409d90ef22108f5890'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:25.499408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da75c29e5faf47409d90ef22108f5890'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:25.505626 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1316/2617] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:57:25 2022] DELETE /identity/v3/users/da75c29e5faf47409d90ef22108f5890 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:25.510289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e813cf-fd62-463f-8623-a1a9ec21e371 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:25.510796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e813cf-fd62-463f-8623-a1a9ec21e371 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:25.510906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e813cf-fd62-463f-8623-a1a9ec21e371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:25.530280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b9e813cf-fd62-463f-8623-a1a9ec21e371 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:25.531457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b9e813cf-fd62-463f-8623-a1a9ec21e371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:25.595006 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1302/2618] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:25.605246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:25.606726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:25.666499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d22a8-20ee-4353-abec-ab50d9b86d51 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:25.671178 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a42b9-ff3c-47d5-a50f-f8d806e7c3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:25.672301 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a42b9-ff3c-47d5-a50f-f8d806e7c3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:25.672833 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a42b9-ff3c-47d5-a50f-f8d806e7c3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:25.673300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a42b9-ff3c-47d5-a50f-f8d806e7c3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:25.684511 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a42b9-ff3c-47d5-a50f-f8d806e7c3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:25.686015 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a42b9-ff3c-47d5-a50f-f8d806e7c3f7 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:25.697445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1317/2619] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:26.072872 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e813cf-fd62-463f-8623-a1a9ec21e371 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:26.074697 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b9e813cf-fd62-463f-8623-a1a9ec21e371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:26.089384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e813cf-fd62-463f-8623-a1a9ec21e371 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:26.095092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:26.096648 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:26.096933 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:26.097384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] PATH_INFO: `/v3/projects/6876653af37247e8a08852f3ad15bd45` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:26.098769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6876653af37247e8a08852f3ad15bd45)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:26.100965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:26.151771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6876653af37247e8a08852f3ad15bd45', 'request_id': 'req-57febd63-abcf-4f46-b3d1-207bf988df2a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.159260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6876653af37247e8a08852f3ad15bd45', 'request_id': 'req-57febd63-abcf-4f46-b3d1-207bf988df2a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.166381 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6876653af37247e8a08852f3ad15bd45', 'request_id': 'req-57febd63-abcf-4f46-b3d1-207bf988df2a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.178289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] The token cache is being invalidate because project 6876653af37247e8a08852f3ad15bd45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:57:26.178605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.179365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.181099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1303/2620] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:57:26 2022] DELETE /identity/v3/projects/6876653af37247e8a08852f3ad15bd45 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:57:26.186257 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a42b9-ff3c-47d5-a50f-f8d806e7c3f7 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:26.197512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a42b9-ff3c-47d5-a50f-f8d806e7c3f7 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:26.280589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:26.281386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:26.281665 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:26.281963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] PATH_INFO: `/v3/users/cd45a0a0c7cc440888ef5541db5fa373` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:26.283231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd45a0a0c7cc440888ef5541db5fa373)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:26.285197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:26.337010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd45a0a0c7cc440888ef5541db5fa373'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.337704 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd45a0a0c7cc440888ef5541db5fa373'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.352484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd45a0a0c7cc440888ef5541db5fa373'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.361253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd45a0a0c7cc440888ef5541db5fa373'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.369663 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-13f5e99f-b93a-4db0-839f-561cc5e89a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd45a0a0c7cc440888ef5541db5fa373'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.376309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1318/2621] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:57:26 2022] DELETE /identity/v3/users/cd45a0a0c7cc440888ef5541db5fa373 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:26.679691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:26.681495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:26.695896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57febd63-abcf-4f46-b3d1-207bf988df2a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:26.758327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:26.759198 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:26.759480 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:26.759755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] PATH_INFO: `/v3/projects/ccf1d3e8fa5947ce869f4f90174e37e4` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:26.761025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf1d3e8fa5947ce869f4f90174e37e4)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:26.762938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:26.823924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf1d3e8fa5947ce869f4f90174e37e4', 'request_id': 'req-9011dbdf-adc3-466e-a634-9f61d8659e06'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.833042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf1d3e8fa5947ce869f4f90174e37e4', 'request_id': 'req-9011dbdf-adc3-466e-a634-9f61d8659e06'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.840449 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf1d3e8fa5947ce869f4f90174e37e4', 'request_id': 'req-9011dbdf-adc3-466e-a634-9f61d8659e06'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.851053 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] The token cache is being invalidate because project ccf1d3e8fa5947ce869f4f90174e37e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:57:26.851325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.852590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9011dbdf-adc3-466e-a634-9f61d8659e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:26.854330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1304/2622] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:57:26 2022] DELETE /identity/v3/projects/ccf1d3e8fa5947ce869f4f90174e37e4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:26.859958 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c07e7-daa5-4466-9811-b7c5d66c8f47 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:26.860251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c07e7-daa5-4466-9811-b7c5d66c8f47 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:26.860559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c07e7-daa5-4466-9811-b7c5d66c8f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:26.881205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d10c07e7-daa5-4466-9811-b7c5d66c8f47 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:26.882446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d10c07e7-daa5-4466-9811-b7c5d66c8f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:26.950367 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1319/2623] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:26.958234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb10ef-6855-4ae9-b9e1-12ce12a3ec4d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:26.958619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb10ef-6855-4ae9-b9e1-12ce12a3ec4d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:26.958908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb10ef-6855-4ae9-b9e1-12ce12a3ec4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:26.978415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c8eb10ef-6855-4ae9-b9e1-12ce12a3ec4d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:26.979754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c8eb10ef-6855-4ae9-b9e1-12ce12a3ec4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:26.987725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1305/2624] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:26.992624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c07e7-daa5-4466-9811-b7c5d66c8f47 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.002774 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c07e7-daa5-4466-9811-b7c5d66c8f47 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.007292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f1ecb-78a1-4ccc-a822-ffababcfd6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.008190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f1ecb-78a1-4ccc-a822-ffababcfd6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.008480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f1ecb-78a1-4ccc-a822-ffababcfd6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.008798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f1ecb-78a1-4ccc-a822-ffababcfd6d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.009532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f1ecb-78a1-4ccc-a822-ffababcfd6d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:57:27.009867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f1ecb-78a1-4ccc-a822-ffababcfd6d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.011479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f1ecb-78a1-4ccc-a822-ffababcfd6d5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.023541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1320/2625] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:57:27.028165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb10ef-6855-4ae9-b9e1-12ce12a3ec4d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.037704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb10ef-6855-4ae9-b9e1-12ce12a3ec4d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.042539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169f2b2-be18-4faf-a394-9c32d713f141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.043208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169f2b2-be18-4faf-a394-9c32d713f141 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.043486 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169f2b2-be18-4faf-a394-9c32d713f141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.043762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169f2b2-be18-4faf-a394-9c32d713f141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.044459 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169f2b2-be18-4faf-a394-9c32d713f141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.046451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169f2b2-be18-4faf-a394-9c32d713f141 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.075803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1306/2626] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:27 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:27.080426 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f1ecb-78a1-4ccc-a822-ffababcfd6d5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.096860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f1ecb-78a1-4ccc-a822-ffababcfd6d5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.159943 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff28b4-5e23-418c-8bff-5f8fd2c65624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.160689 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff28b4-5e23-418c-8bff-5f8fd2c65624 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.160955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff28b4-5e23-418c-8bff-5f8fd2c65624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.161281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff28b4-5e23-418c-8bff-5f8fd2c65624 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.161980 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff28b4-5e23-418c-8bff-5f8fd2c65624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.163911 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff28b4-5e23-418c-8bff-5f8fd2c65624 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.185088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1321/2627] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:27 2022] POST /identity/v3/users => generated 282 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:27.189700 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169f2b2-be18-4faf-a394-9c32d713f141 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.200351 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169f2b2-be18-4faf-a394-9c32d713f141 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.205404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231eaf4-b833-4997-ad31-439622e1c831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.206060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231eaf4-b833-4997-ad31-439622e1c831 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.206338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231eaf4-b833-4997-ad31-439622e1c831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.206609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231eaf4-b833-4997-ad31-439622e1c831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.207327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231eaf4-b833-4997-ad31-439622e1c831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.208963 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231eaf4-b833-4997-ad31-439622e1c831 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.219172 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1307/2628] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:27.223158 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff28b4-5e23-418c-8bff-5f8fd2c65624 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.233202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff28b4-5e23-418c-8bff-5f8fd2c65624 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.237817 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18cb8a-0d71-46c5-b4ce-8fab15f1a305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.238476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18cb8a-0d71-46c5-b4ce-8fab15f1a305 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.238741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18cb8a-0d71-46c5-b4ce-8fab15f1a305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.239006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18cb8a-0d71-46c5-b4ce-8fab15f1a305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.239726 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18cb8a-0d71-46c5-b4ce-8fab15f1a305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.241358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18cb8a-0d71-46c5-b4ce-8fab15f1a305 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.251731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1322/2629] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:27.255916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231eaf4-b833-4997-ad31-439622e1c831 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.266824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231eaf4-b833-4997-ad31-439622e1c831 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.273134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8a116-0018-4877-9e57-8c0287cb990f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.274035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8a116-0018-4877-9e57-8c0287cb990f admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.274342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8a116-0018-4877-9e57-8c0287cb990f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.274593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8a116-0018-4877-9e57-8c0287cb990f admin admin] PATH_INFO: `/v3/projects/d39aa07ede7e43cba8974b1982f983b6/users/8ba6a164476e4a78a1cb229816eb2884/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.290202 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8a116-0018-4877-9e57-8c0287cb990f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39aa07ede7e43cba8974b1982f983b6, user_id=8ba6a164476e4a78a1cb229816eb2884, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.292870 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8a116-0018-4877-9e57-8c0287cb990f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.307401 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1308/2630] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:27 2022] PUT /identity/v3/projects/d39aa07ede7e43cba8974b1982f983b6/users/8ba6a164476e4a78a1cb229816eb2884/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:27.319768 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae71613-34ff-464b-894c-07b4323d64d7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.320156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae71613-34ff-464b-894c-07b4323d64d7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.320384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae71613-34ff-464b-894c-07b4323d64d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.346919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5ae71613-34ff-464b-894c-07b4323d64d7 None None] MFA Rules not processed for user `8ba6a164476e4a78a1cb229816eb2884`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:27.348050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5ae71613-34ff-464b-894c-07b4323d64d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:27.421507 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1323/2631] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:27 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:27.426571 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8a116-0018-4877-9e57-8c0287cb990f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.492949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8a116-0018-4877-9e57-8c0287cb990f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.498289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8b382-622e-467f-ab0d-63e1129de14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.498957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8b382-622e-467f-ab0d-63e1129de14b admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.499241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8b382-622e-467f-ab0d-63e1129de14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.499508 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8b382-622e-467f-ab0d-63e1129de14b admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.500189 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8b382-622e-467f-ab0d-63e1129de14b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.502149 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8b382-622e-467f-ab0d-63e1129de14b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.531104 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1309/2632] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:27 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:27.537464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae71613-34ff-464b-894c-07b4323d64d7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.549350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae71613-34ff-464b-894c-07b4323d64d7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.615551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f4bb3-f1cb-427b-9c3b-65acf2c9c20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.616309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f4bb3-f1cb-427b-9c3b-65acf2c9c20b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.616728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f4bb3-f1cb-427b-9c3b-65acf2c9c20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.617091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f4bb3-f1cb-427b-9c3b-65acf2c9c20b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.622215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f4bb3-f1cb-427b-9c3b-65acf2c9c20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.622215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f4bb3-f1cb-427b-9c3b-65acf2c9c20b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.642556 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1324/2633] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:27 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:27.646619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8b382-622e-467f-ab0d-63e1129de14b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.657330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8b382-622e-467f-ab0d-63e1129de14b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.662073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ca0f6-4042-4bad-acff-b2e4641c18d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.662728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ca0f6-4042-4bad-acff-b2e4641c18d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.663004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ca0f6-4042-4bad-acff-b2e4641c18d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.663271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ca0f6-4042-4bad-acff-b2e4641c18d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.664016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ca0f6-4042-4bad-acff-b2e4641c18d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.665965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ca0f6-4042-4bad-acff-b2e4641c18d8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.676188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1310/2634] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:27.680458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f4bb3-f1cb-427b-9c3b-65acf2c9c20b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.691438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f4bb3-f1cb-427b-9c3b-65acf2c9c20b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.697457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3006a34-98fe-4b67-b01f-ffc2c631ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.698339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006a34-98fe-4b67-b01f-ffc2c631ffde admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.698618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006a34-98fe-4b67-b01f-ffc2c631ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.698889 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006a34-98fe-4b67-b01f-ffc2c631ffde admin admin] PATH_INFO: `/v3/projects/290fd96a7864494ab75e2774e95a124b/users/f0b641fd8c6043d5969fd7fccabefb6a/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.714974 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3006a34-98fe-4b67-b01f-ffc2c631ffde admin admin] RBAC: Authorizing `identity:create_grant(project_id=290fd96a7864494ab75e2774e95a124b, user_id=f0b641fd8c6043d5969fd7fccabefb6a, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.717639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3006a34-98fe-4b67-b01f-ffc2c631ffde admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.733126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1325/2635] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:27 2022] PUT /identity/v3/projects/290fd96a7864494ab75e2774e95a124b/users/f0b641fd8c6043d5969fd7fccabefb6a/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:27.737754 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ca0f6-4042-4bad-acff-b2e4641c18d8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.749952 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ca0f6-4042-4bad-acff-b2e4641c18d8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.812084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c90970-d069-4285-ab09-f4ca2fa51b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.813134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90970-d069-4285-ab09-f4ca2fa51b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.813407 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90970-d069-4285-ab09-f4ca2fa51b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.813662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c90970-d069-4285-ab09-f4ca2fa51b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.814393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c90970-d069-4285-ab09-f4ca2fa51b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.815987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c90970-d069-4285-ab09-f4ca2fa51b79 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.825874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1311/2636] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:27.830129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3006a34-98fe-4b67-b01f-ffc2c631ffde admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.840753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3006a34-98fe-4b67-b01f-ffc2c631ffde admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:27.845940 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e0b61-eba8-4042-8cbf-8e608557ec30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:27.846811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e0b61-eba8-4042-8cbf-8e608557ec30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.847080 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e0b61-eba8-4042-8cbf-8e608557ec30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.847373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e0b61-eba8-4042-8cbf-8e608557ec30 admin admin] PATH_INFO: `/v3/projects/290fd96a7864494ab75e2774e95a124b/users/f0b641fd8c6043d5969fd7fccabefb6a/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.849407 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e0b61-eba8-4042-8cbf-8e608557ec30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=290fd96a7864494ab75e2774e95a124b, user_id=f0b641fd8c6043d5969fd7fccabefb6a, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:27.852024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e0b61-eba8-4042-8cbf-8e608557ec30 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:27.862420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1326/2637] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:27 2022] PUT /identity/v3/projects/290fd96a7864494ab75e2774e95a124b/users/f0b641fd8c6043d5969fd7fccabefb6a/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:27.872441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aec4a-be31-4d29-b7d6-658ba86d3ca5 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:27.872657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aec4a-be31-4d29-b7d6-658ba86d3ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:27.872946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1aec4a-be31-4d29-b7d6-658ba86d3ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:27.902661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ad1aec4a-be31-4d29-b7d6-658ba86d3ca5 None None] MFA Rules not processed for user `f0b641fd8c6043d5969fd7fccabefb6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:27.903867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ad1aec4a-be31-4d29-b7d6-658ba86d3ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:27.977898 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1312/2638] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:27 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:27.992116 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e0b61-eba8-4042-8cbf-8e608557ec30 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:27.994313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ed9e0b61-eba8-4042-8cbf-8e608557ec30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:28.054247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e0b61-eba8-4042-8cbf-8e608557ec30 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:28.058620 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f38ab-83c8-4b78-89f3-81c579a7e0cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:28.059265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f38ab-83c8-4b78-89f3-81c579a7e0cc service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:28.059541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f38ab-83c8-4b78-89f3-81c579a7e0cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:28.059811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f38ab-83c8-4b78-89f3-81c579a7e0cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:28.069798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f38ab-83c8-4b78-89f3-81c579a7e0cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:28.071317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f38ab-83c8-4b78-89f3-81c579a7e0cc service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:28.080662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1327/2639] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:27 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:28.161569 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aec4a-be31-4d29-b7d6-658ba86d3ca5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:28.172070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1aec4a-be31-4d29-b7d6-658ba86d3ca5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:28.175775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698506c-fd80-44c6-a9c6-1c616897b109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:28.176426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698506c-fd80-44c6-a9c6-1c616897b109 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:28.176763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698506c-fd80-44c6-a9c6-1c616897b109 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:28.177033 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698506c-fd80-44c6-a9c6-1c616897b109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:28.234372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2698506c-fd80-44c6-a9c6-1c616897b109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:28.235812 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2698506c-fd80-44c6-a9c6-1c616897b109 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:28.244932 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1313/2640] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:28 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:30.991028 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f38ab-83c8-4b78-89f3-81c579a7e0cc service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:31.060574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f38ab-83c8-4b78-89f3-81c579a7e0cc service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:31.067849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:31.069398 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:31.069851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:31.070298 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] PATH_INFO: `/v3/users/8ba6a164476e4a78a1cb229816eb2884` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:31.071730 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba6a164476e4a78a1cb229816eb2884)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:31.073686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:31.132060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba6a164476e4a78a1cb229816eb2884'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.133063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba6a164476e4a78a1cb229816eb2884'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.145933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba6a164476e4a78a1cb229816eb2884'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.154513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba6a164476e4a78a1cb229816eb2884'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.165633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba6a164476e4a78a1cb229816eb2884'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.171839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1328/2641] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:57:30 2022] DELETE /identity/v3/users/8ba6a164476e4a78a1cb229816eb2884 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:31.176531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa6f5c-3484-4740-a5be-b6145d879934 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:31.177054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa6f5c-3484-4740-a5be-b6145d879934 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:31.177168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa6f5c-3484-4740-a5be-b6145d879934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:31.196368 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bdaa6f5c-3484-4740-a5be-b6145d879934 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:31.197607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bdaa6f5c-3484-4740-a5be-b6145d879934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:31.257536 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1314/2642] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:31.270481 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:31.272136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:31.338236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3a91d-45c3-4ac2-899c-07412bcbc42f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:31.342910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ddebe-35ad-47ca-b1da-d03a1068041c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:31.343566 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ddebe-35ad-47ca-b1da-d03a1068041c service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:31.343846 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ddebe-35ad-47ca-b1da-d03a1068041c service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:31.344115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ddebe-35ad-47ca-b1da-d03a1068041c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:31.353744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ddebe-35ad-47ca-b1da-d03a1068041c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:31.355176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ddebe-35ad-47ca-b1da-d03a1068041c service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:31.365020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1329/2643] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:31.721126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa6f5c-3484-4740-a5be-b6145d879934 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:31.722651 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bdaa6f5c-3484-4740-a5be-b6145d879934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:31.734979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa6f5c-3484-4740-a5be-b6145d879934 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:31.741146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:31.742214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:31.742497 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:31.742771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] PATH_INFO: `/v3/projects/d39aa07ede7e43cba8974b1982f983b6` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:31.744223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d39aa07ede7e43cba8974b1982f983b6)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:31.746246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:31.796763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd39aa07ede7e43cba8974b1982f983b6', 'request_id': 'req-615d2885-8b5b-49b6-8100-6c4d991dd599'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.805241 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd39aa07ede7e43cba8974b1982f983b6', 'request_id': 'req-615d2885-8b5b-49b6-8100-6c4d991dd599'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.811634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd39aa07ede7e43cba8974b1982f983b6', 'request_id': 'req-615d2885-8b5b-49b6-8100-6c4d991dd599'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.823206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] The token cache is being invalidate because project d39aa07ede7e43cba8974b1982f983b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:57:31.823494 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.824239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.825805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1315/2644] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:57:31 2022] DELETE /identity/v3/projects/d39aa07ede7e43cba8974b1982f983b6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:31.829780 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ddebe-35ad-47ca-b1da-d03a1068041c service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:31.839625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ddebe-35ad-47ca-b1da-d03a1068041c service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:31.902363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:31.903243 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:31.903522 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:31.903794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] PATH_INFO: `/v3/users/f0b641fd8c6043d5969fd7fccabefb6a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:31.905119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0b641fd8c6043d5969fd7fccabefb6a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:31.906949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:31.955181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0b641fd8c6043d5969fd7fccabefb6a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.955919 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0b641fd8c6043d5969fd7fccabefb6a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.968844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0b641fd8c6043d5969fd7fccabefb6a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.978365 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0b641fd8c6043d5969fd7fccabefb6a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.985082 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18c0547d-38f9-4283-85dd-bb1ccfc6e11b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0b641fd8c6043d5969fd7fccabefb6a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:31.992559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1330/2645] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:57:31 2022] DELETE /identity/v3/users/f0b641fd8c6043d5969fd7fccabefb6a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:32.272191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:32.273810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:32.286834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d2885-8b5b-49b6-8100-6c4d991dd599 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:32.343670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:32.344679 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:32.344965 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:32.345240 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] PATH_INFO: `/v3/projects/290fd96a7864494ab75e2774e95a124b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:32.346463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=290fd96a7864494ab75e2774e95a124b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:32.348359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:32.394480 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '290fd96a7864494ab75e2774e95a124b', 'request_id': 'req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:32.401741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '290fd96a7864494ab75e2774e95a124b', 'request_id': 'req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:32.408160 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '290fd96a7864494ab75e2774e95a124b', 'request_id': 'req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:32.423007 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] The token cache is being invalidate because project 290fd96a7864494ab75e2774e95a124b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:57:32.423279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:32.423978 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6d07b677-c45d-4cc9-a0eb-6fa51798b0f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:32.425708 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1316/2646] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:57:32 2022] DELETE /identity/v3/projects/290fd96a7864494ab75e2774e95a124b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:32.431851 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0dc1d-a969-4b10-b131-25ed8d321939 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:32.432071 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0dc1d-a969-4b10-b131-25ed8d321939 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:32.432386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0dc1d-a969-4b10-b131-25ed8d321939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:32.457724 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fcb0dc1d-a969-4b10-b131-25ed8d321939 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:32.458939 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fcb0dc1d-a969-4b10-b131-25ed8d321939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:32.521964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1331/2647] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:32.529996 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1183b80-303a-4e8c-ab09-b1a9a45bd1ec None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:32.530286 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1183b80-303a-4e8c-ab09-b1a9a45bd1ec None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:32.530561 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1183b80-303a-4e8c-ab09-b1a9a45bd1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:32.549352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a1183b80-303a-4e8c-ab09-b1a9a45bd1ec None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:32.550547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a1183b80-303a-4e8c-ab09-b1a9a45bd1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:32.560042 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1317/2648] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:32.564734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0dc1d-a969-4b10-b131-25ed8d321939 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:32.574809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0dc1d-a969-4b10-b131-25ed8d321939 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:32.579751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3cdf9-dfa4-4f3b-b586-529919c697ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:32.580408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3cdf9-dfa4-4f3b-b586-529919c697ac admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:32.580716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3cdf9-dfa4-4f3b-b586-529919c697ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:32.580988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3cdf9-dfa4-4f3b-b586-529919c697ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:32.581709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3cdf9-dfa4-4f3b-b586-529919c697ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:57:32.582041 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3cdf9-dfa4-4f3b-b586-529919c697ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:32.583627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3cdf9-dfa4-4f3b-b586-529919c697ac admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:32.595251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1332/2649] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:57:32.599733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1183b80-303a-4e8c-ab09-b1a9a45bd1ec None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:32.609552 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1183b80-303a-4e8c-ab09-b1a9a45bd1ec None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:32.615203 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ae7c1-6880-4d47-9795-da6a2ea70167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:32.615866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ae7c1-6880-4d47-9795-da6a2ea70167 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:32.616148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ae7c1-6880-4d47-9795-da6a2ea70167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:32.616421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ae7c1-6880-4d47-9795-da6a2ea70167 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:32.617180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ae7c1-6880-4d47-9795-da6a2ea70167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:32.620446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ae7c1-6880-4d47-9795-da6a2ea70167 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:32.650687 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1318/2650] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:32 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:32.651963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3cdf9-dfa4-4f3b-b586-529919c697ac admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:32.661991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3cdf9-dfa4-4f3b-b586-529919c697ac admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:32.720655 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601f36e-e7d7-4e59-a4b6-822af0ae1c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:32.721420 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601f36e-e7d7-4e59-a4b6-822af0ae1c5c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:32.721779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601f36e-e7d7-4e59-a4b6-822af0ae1c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:32.723476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601f36e-e7d7-4e59-a4b6-822af0ae1c5c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:32.723476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601f36e-e7d7-4e59-a4b6-822af0ae1c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:32.724960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601f36e-e7d7-4e59-a4b6-822af0ae1c5c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:32.750995 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1333/2651] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:32 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:32.755250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ae7c1-6880-4d47-9795-da6a2ea70167 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:32.766466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ae7c1-6880-4d47-9795-da6a2ea70167 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:32.771158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368736b-c19f-4d78-a915-40c8a65eb935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:32.771821 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368736b-c19f-4d78-a915-40c8a65eb935 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:32.772118 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368736b-c19f-4d78-a915-40c8a65eb935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:32.772366 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368736b-c19f-4d78-a915-40c8a65eb935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:32.773121 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368736b-c19f-4d78-a915-40c8a65eb935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:32.774827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368736b-c19f-4d78-a915-40c8a65eb935 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:32.785625 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1319/2652] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:32.789760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601f36e-e7d7-4e59-a4b6-822af0ae1c5c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:32.799950 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601f36e-e7d7-4e59-a4b6-822af0ae1c5c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:32.804735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941a3ab-1ac6-4f58-838e-1e12ce1e52de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:32.806260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941a3ab-1ac6-4f58-838e-1e12ce1e52de admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:32.806552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941a3ab-1ac6-4f58-838e-1e12ce1e52de admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:32.806823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941a3ab-1ac6-4f58-838e-1e12ce1e52de admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:32.807592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941a3ab-1ac6-4f58-838e-1e12ce1e52de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:32.809213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941a3ab-1ac6-4f58-838e-1e12ce1e52de admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:32.821047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1334/2653] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:32.825273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368736b-c19f-4d78-a915-40c8a65eb935 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:32.835566 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368736b-c19f-4d78-a915-40c8a65eb935 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:32.841044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947cd14-e556-4609-a96d-8530a932c938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:32.842070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947cd14-e556-4609-a96d-8530a932c938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:32.842352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947cd14-e556-4609-a96d-8530a932c938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:32.842614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947cd14-e556-4609-a96d-8530a932c938 admin admin] PATH_INFO: `/v3/projects/b80fca47418b426683056a565d4ba692/users/ea91ff94ccc44fb9bb91288a3e598ab6/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:32.857505 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947cd14-e556-4609-a96d-8530a932c938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80fca47418b426683056a565d4ba692, user_id=ea91ff94ccc44fb9bb91288a3e598ab6, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:32.860349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947cd14-e556-4609-a96d-8530a932c938 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:32.874465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1320/2654] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:32 2022] PUT /identity/v3/projects/b80fca47418b426683056a565d4ba692/users/ea91ff94ccc44fb9bb91288a3e598ab6/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:57:32.885540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d16f-fcc1-41bb-96cd-13960ebb376e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:32.885834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d16f-fcc1-41bb-96cd-13960ebb376e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:32.886114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d16f-fcc1-41bb-96cd-13960ebb376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:32.912541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-59a3d16f-fcc1-41bb-96cd-13960ebb376e None None] MFA Rules not processed for user `ea91ff94ccc44fb9bb91288a3e598ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:32.913811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-59a3d16f-fcc1-41bb-96cd-13960ebb376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:32.983183 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1335/2655] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:32.987675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947cd14-e556-4609-a96d-8530a932c938 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.047816 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947cd14-e556-4609-a96d-8530a932c938 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.053442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b0230-3573-4cad-baa8-19677d506b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.054108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b0230-3573-4cad-baa8-19677d506b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.054387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b0230-3573-4cad-baa8-19677d506b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.054653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b0230-3573-4cad-baa8-19677d506b57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.055501 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b0230-3573-4cad-baa8-19677d506b57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.057682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b0230-3573-4cad-baa8-19677d506b57 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.085394 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1321/2656] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:32 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:33.089399 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d16f-fcc1-41bb-96cd-13960ebb376e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.100172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d16f-fcc1-41bb-96cd-13960ebb376e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.166749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d5339-cad6-4d16-8c41-87ce403883a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.167434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d5339-cad6-4d16-8c41-87ce403883a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.167709 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d5339-cad6-4d16-8c41-87ce403883a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.167964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d5339-cad6-4d16-8c41-87ce403883a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.168703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d5339-cad6-4d16-8c41-87ce403883a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.172330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d5339-cad6-4d16-8c41-87ce403883a8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.195899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1336/2657] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:33 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:57:33.197621 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b0230-3573-4cad-baa8-19677d506b57 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.208125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b0230-3573-4cad-baa8-19677d506b57 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.213593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587ea47-2b28-4661-9382-bf8e050aa1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.214414 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587ea47-2b28-4661-9382-bf8e050aa1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.214541 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587ea47-2b28-4661-9382-bf8e050aa1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.214805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587ea47-2b28-4661-9382-bf8e050aa1c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.215535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587ea47-2b28-4661-9382-bf8e050aa1c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.217234 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587ea47-2b28-4661-9382-bf8e050aa1c7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.228086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1322/2658] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:33.232736 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d5339-cad6-4d16-8c41-87ce403883a8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.244366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d5339-cad6-4d16-8c41-87ce403883a8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.249158 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380cc3d-64b1-4f2f-8a1b-648b86e97caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.249814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380cc3d-64b1-4f2f-8a1b-648b86e97caf admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.250090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380cc3d-64b1-4f2f-8a1b-648b86e97caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.250356 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380cc3d-64b1-4f2f-8a1b-648b86e97caf admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.251064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380cc3d-64b1-4f2f-8a1b-648b86e97caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.252704 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380cc3d-64b1-4f2f-8a1b-648b86e97caf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.266542 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1337/2659] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:33.268111 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587ea47-2b28-4661-9382-bf8e050aa1c7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.279253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587ea47-2b28-4661-9382-bf8e050aa1c7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.283843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9d44-1151-4c1b-878e-a89c74695739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.284758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9d44-1151-4c1b-878e-a89c74695739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.285037 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9d44-1151-4c1b-878e-a89c74695739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.285304 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9d44-1151-4c1b-878e-a89c74695739 admin admin] PATH_INFO: `/v3/projects/3630fd9253e14696ac1a3e3325d35a7f/users/f02b905137064385925a6d17ea3d27dc/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.302666 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c9d44-1151-4c1b-878e-a89c74695739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3630fd9253e14696ac1a3e3325d35a7f, user_id=f02b905137064385925a6d17ea3d27dc, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.306120 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c9d44-1151-4c1b-878e-a89c74695739 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.321861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1323/2660] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:33 2022] PUT /identity/v3/projects/3630fd9253e14696ac1a3e3325d35a7f/users/f02b905137064385925a6d17ea3d27dc/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:57:33.332206 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a388db-4189-45a7-9d62-e6294273ed0e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.332519 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a388db-4189-45a7-9d62-e6294273ed0e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.332809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a388db-4189-45a7-9d62-e6294273ed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.359975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-87a388db-4189-45a7-9d62-e6294273ed0e None None] MFA Rules not processed for user `f02b905137064385925a6d17ea3d27dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:33.361207 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-87a388db-4189-45a7-9d62-e6294273ed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:33.421951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1338/2661] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:57:33.427096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9d44-1151-4c1b-878e-a89c74695739 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.492287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9d44-1151-4c1b-878e-a89c74695739 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.497186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b799eae-0779-4b74-991e-c272e42fce8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.497828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b799eae-0779-4b74-991e-c272e42fce8e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.498108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b799eae-0779-4b74-991e-c272e42fce8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.498378 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b799eae-0779-4b74-991e-c272e42fce8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.499070 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b799eae-0779-4b74-991e-c272e42fce8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.501009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b799eae-0779-4b74-991e-c272e42fce8e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.531235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1324/2662] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:33 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:33.535638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a388db-4189-45a7-9d62-e6294273ed0e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.546949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a388db-4189-45a7-9d62-e6294273ed0e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.611773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46bedb-7a78-41d0-9f4e-dca451795c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.612458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46bedb-7a78-41d0-9f4e-dca451795c4f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.612771 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46bedb-7a78-41d0-9f4e-dca451795c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.613043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46bedb-7a78-41d0-9f4e-dca451795c4f admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.613747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46bedb-7a78-41d0-9f4e-dca451795c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.615672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46bedb-7a78-41d0-9f4e-dca451795c4f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.639350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1339/2663] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:33 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:57:33.643826 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b799eae-0779-4b74-991e-c272e42fce8e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.655986 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b799eae-0779-4b74-991e-c272e42fce8e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.661834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e212f92-9961-4dde-ae5d-3c849c1f6540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.662530 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e212f92-9961-4dde-ae5d-3c849c1f6540 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.662804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e212f92-9961-4dde-ae5d-3c849c1f6540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.663066 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e212f92-9961-4dde-ae5d-3c849c1f6540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.663791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e212f92-9961-4dde-ae5d-3c849c1f6540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.665444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e212f92-9961-4dde-ae5d-3c849c1f6540 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.676657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1325/2664] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:33.681231 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46bedb-7a78-41d0-9f4e-dca451795c4f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.691727 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46bedb-7a78-41d0-9f4e-dca451795c4f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.696509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e431a-5312-4389-bca6-3027dcb69b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.697405 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e431a-5312-4389-bca6-3027dcb69b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.697681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e431a-5312-4389-bca6-3027dcb69b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.697950 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e431a-5312-4389-bca6-3027dcb69b3a admin admin] PATH_INFO: `/v3/projects/58cea02f0a9f420489e9e98ce47f74c8/users/aa466fe69c4742ab9024382459e583d8/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.714899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e431a-5312-4389-bca6-3027dcb69b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58cea02f0a9f420489e9e98ce47f74c8, user_id=aa466fe69c4742ab9024382459e583d8, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.717644 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e431a-5312-4389-bca6-3027dcb69b3a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.732268 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1340/2665] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:33 2022] PUT /identity/v3/projects/58cea02f0a9f420489e9e98ce47f74c8/users/aa466fe69c4742ab9024382459e583d8/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:57:33.736800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e212f92-9961-4dde-ae5d-3c849c1f6540 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.747223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e212f92-9961-4dde-ae5d-3c849c1f6540 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.803179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe0931-9a4b-4b20-ad12-f7eb347c0067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.803843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe0931-9a4b-4b20-ad12-f7eb347c0067 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.805073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe0931-9a4b-4b20-ad12-f7eb347c0067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.805412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe0931-9a4b-4b20-ad12-f7eb347c0067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.806131 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe0931-9a4b-4b20-ad12-f7eb347c0067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.807740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe0931-9a4b-4b20-ad12-f7eb347c0067 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.819597 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1326/2666] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:33.826690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e431a-5312-4389-bca6-3027dcb69b3a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.840289 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e431a-5312-4389-bca6-3027dcb69b3a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:33.845295 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e040000-3293-4063-a120-2f47bf65e607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:33.846181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e040000-3293-4063-a120-2f47bf65e607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.846464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e040000-3293-4063-a120-2f47bf65e607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.846742 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e040000-3293-4063-a120-2f47bf65e607 admin admin] PATH_INFO: `/v3/projects/58cea02f0a9f420489e9e98ce47f74c8/users/aa466fe69c4742ab9024382459e583d8/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.848827 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e040000-3293-4063-a120-2f47bf65e607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58cea02f0a9f420489e9e98ce47f74c8, user_id=aa466fe69c4742ab9024382459e583d8, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:33.851447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e040000-3293-4063-a120-2f47bf65e607 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:33.861685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1341/2667] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:33 2022] PUT /identity/v3/projects/58cea02f0a9f420489e9e98ce47f74c8/users/aa466fe69c4742ab9024382459e583d8/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:33.872287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ea265-47d3-46d6-9d10-b5413451169e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:33.872605 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ea265-47d3-46d6-9d10-b5413451169e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:33.872880 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ea265-47d3-46d6-9d10-b5413451169e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:33.898191 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c43ea265-47d3-46d6-9d10-b5413451169e None None] MFA Rules not processed for user `aa466fe69c4742ab9024382459e583d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:33.899465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c43ea265-47d3-46d6-9d10-b5413451169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:33.974158 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1327/2668] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:57:33.989955 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e040000-3293-4063-a120-2f47bf65e607 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:33.992135 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9e040000-3293-4063-a120-2f47bf65e607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:34.073893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e040000-3293-4063-a120-2f47bf65e607 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:34.078305 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9c4ef-5510-4fcf-9b29-81beb97cd3f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:34.079002 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9c4ef-5510-4fcf-9b29-81beb97cd3f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:34.079285 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9c4ef-5510-4fcf-9b29-81beb97cd3f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:34.079554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9c4ef-5510-4fcf-9b29-81beb97cd3f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:34.089729 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9c4ef-5510-4fcf-9b29-81beb97cd3f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:34.091460 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9c4ef-5510-4fcf-9b29-81beb97cd3f8 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:34.103109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1342/2669] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:34.633579 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ea265-47d3-46d6-9d10-b5413451169e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:34.635221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c43ea265-47d3-46d6-9d10-b5413451169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:34.648125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ea265-47d3-46d6-9d10-b5413451169e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:34.652128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f4b42-0108-447f-8ac9-a54d4574e88e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:34.652839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f4b42-0108-447f-8ac9-a54d4574e88e service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:34.653113 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f4b42-0108-447f-8ac9-a54d4574e88e service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:34.653404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f4b42-0108-447f-8ac9-a54d4574e88e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:34.717196 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f4b42-0108-447f-8ac9-a54d4574e88e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:34.718705 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f4b42-0108-447f-8ac9-a54d4574e88e service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:34.729481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1328/2670] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:43.177054 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf963ac9-fe4d-4af1-a637-1a09e3262e69 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:43.177575 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf963ac9-fe4d-4af1-a637-1a09e3262e69 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:43.177575 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf963ac9-fe4d-4af1-a637-1a09e3262e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:43.238119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bf963ac9-fe4d-4af1-a637-1a09e3262e69 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:43.239311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bf963ac9-fe4d-4af1-a637-1a09e3262e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:43.314121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1343/2671] 10.222.0.50 () {58 vars in 1009 bytes} [Fri Apr 29 18:57:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:57:43.364258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f4b42-0108-447f-8ac9-a54d4574e88e service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:43.365900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f09f4b42-0108-447f-8ac9-a54d4574e88e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:43.459135 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f4b42-0108-447f-8ac9-a54d4574e88e service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:43.465670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce6feb-7127-4984-bb7e-7fa3c59b0317 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:43.466322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce6feb-7127-4984-bb7e-7fa3c59b0317 service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:43.466608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce6feb-7127-4984-bb7e-7fa3c59b0317 service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:43.466871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce6feb-7127-4984-bb7e-7fa3c59b0317 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:43.475520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce6feb-7127-4984-bb7e-7fa3c59b0317 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:43.477132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce6feb-7127-4984-bb7e-7fa3c59b0317 service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:43.485336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1329/2672] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:57:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:57:45.175166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf963ac9-fe4d-4af1-a637-1a09e3262e69 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:45.186514 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf963ac9-fe4d-4af1-a637-1a09e3262e69 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:45.191694 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80282a2-f645-4485-bf06-81d4d3df8fd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:45.192608 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80282a2-f645-4485-bf06-81d4d3df8fd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:45.192893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80282a2-f645-4485-bf06-81d4d3df8fd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:45.193142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80282a2-f645-4485-bf06-81d4d3df8fd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:45.263640 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80282a2-f645-4485-bf06-81d4d3df8fd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:45.265368 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80282a2-f645-4485-bf06-81d4d3df8fd9 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:45.275501 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1344/2673] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:47.519239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce6feb-7127-4984-bb7e-7fa3c59b0317 service swift] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:47.591688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce6feb-7127-4984-bb7e-7fa3c59b0317 service swift] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:47.597529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:47.598441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:47.598726 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:47.599000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] PATH_INFO: `/v3/users/ea91ff94ccc44fb9bb91288a3e598ab6` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:47.600326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea91ff94ccc44fb9bb91288a3e598ab6)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:47.603309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:47.661892 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea91ff94ccc44fb9bb91288a3e598ab6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:47.662656 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea91ff94ccc44fb9bb91288a3e598ab6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:47.680296 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea91ff94ccc44fb9bb91288a3e598ab6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:47.690837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea91ff94ccc44fb9bb91288a3e598ab6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:47.700515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea91ff94ccc44fb9bb91288a3e598ab6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:47.707766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1330/2674] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:57:47 2022] DELETE /identity/v3/users/ea91ff94ccc44fb9bb91288a3e598ab6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:47.712358 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c1b4f-b462-460e-8dd3-98bb869c7950 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:47.712883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c1b4f-b462-460e-8dd3-98bb869c7950 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:47.713005 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c1b4f-b462-460e-8dd3-98bb869c7950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:47.732372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-987c1b4f-b462-460e-8dd3-98bb869c7950 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:47.733861 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-987c1b4f-b462-460e-8dd3-98bb869c7950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:47.796636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1345/2675] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:47.809365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:47.813741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:47.890984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d63704-eb9b-4d8d-8493-59660abb4c6d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:47.895193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadf19c-d582-4f13-914a-724a87c94b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:47.896048 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf19c-d582-4f13-914a-724a87c94b57 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:47.896545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf19c-d582-4f13-914a-724a87c94b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:47.897032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf19c-d582-4f13-914a-724a87c94b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:47.908088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadf19c-d582-4f13-914a-724a87c94b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:47.910047 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadf19c-d582-4f13-914a-724a87c94b57 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:47.923654 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1331/2676] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:48.225314 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c1b4f-b462-460e-8dd3-98bb869c7950 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:48.226897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-987c1b4f-b462-460e-8dd3-98bb869c7950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:48.248335 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c1b4f-b462-460e-8dd3-98bb869c7950 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:48.256638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:48.257512 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:48.258014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:48.258290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] PATH_INFO: `/v3/projects/b80fca47418b426683056a565d4ba692` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:48.265588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b80fca47418b426683056a565d4ba692)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:48.271643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:48.339979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b80fca47418b426683056a565d4ba692', 'request_id': 'req-63f9bca1-0da9-4423-a85c-12fced35479f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:48.349053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b80fca47418b426683056a565d4ba692', 'request_id': 'req-63f9bca1-0da9-4423-a85c-12fced35479f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:48.356684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b80fca47418b426683056a565d4ba692', 'request_id': 'req-63f9bca1-0da9-4423-a85c-12fced35479f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:48.368205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] The token cache is being invalidate because project b80fca47418b426683056a565d4ba692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:57:48.368524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:48.369269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:48.371036 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1346/2677] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:57:48 2022] DELETE /identity/v3/projects/b80fca47418b426683056a565d4ba692 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:48.375206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadf19c-d582-4f13-914a-724a87c94b57 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:48.386093 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadf19c-d582-4f13-914a-724a87c94b57 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:48.445998 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:48.446936 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:48.447258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:48.447548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] PATH_INFO: `/v3/users/f02b905137064385925a6d17ea3d27dc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:48.448823 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f02b905137064385925a6d17ea3d27dc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:48.450972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:48.503081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f02b905137064385925a6d17ea3d27dc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:48.503848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f02b905137064385925a6d17ea3d27dc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:48.516476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f02b905137064385925a6d17ea3d27dc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:48.523531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f02b905137064385925a6d17ea3d27dc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:48.532834 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f02b905137064385925a6d17ea3d27dc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:48.539960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1332/2678] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:57:48 2022] DELETE /identity/v3/users/f02b905137064385925a6d17ea3d27dc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:48.881784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:48.885852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:48.917812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9bca1-0da9-4423-a85c-12fced35479f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:48.984191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:48.985375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:48.985776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:48.985989 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] PATH_INFO: `/v3/projects/3630fd9253e14696ac1a3e3325d35a7f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:48.987255 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3630fd9253e14696ac1a3e3325d35a7f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:48.989366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:49.053691 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3630fd9253e14696ac1a3e3325d35a7f', 'request_id': 'req-cc1ebf97-81da-4cdd-a99a-a35985bbc764'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.065221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3630fd9253e14696ac1a3e3325d35a7f', 'request_id': 'req-cc1ebf97-81da-4cdd-a99a-a35985bbc764'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.076235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3630fd9253e14696ac1a3e3325d35a7f', 'request_id': 'req-cc1ebf97-81da-4cdd-a99a-a35985bbc764'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.093743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] The token cache is being invalidate because project 3630fd9253e14696ac1a3e3325d35a7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:57:49.094130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.095040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.100409 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1347/2679] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:57:48 2022] DELETE /identity/v3/projects/3630fd9253e14696ac1a3e3325d35a7f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:49.101403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:49.102818 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:49.117360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929515d1-c455-4b36-b337-77067b01b6cb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:49.187940 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:49.189173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:49.189673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:49.190721 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] PATH_INFO: `/v3/users/aa466fe69c4742ab9024382459e583d8` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:49.191938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa466fe69c4742ab9024382459e583d8)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:49.196142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:49.246836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa466fe69c4742ab9024382459e583d8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.247974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa466fe69c4742ab9024382459e583d8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.259817 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa466fe69c4742ab9024382459e583d8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.266565 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa466fe69c4742ab9024382459e583d8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.275695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-80f71e67-c47d-4208-8c0c-ad3c3e990bee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa466fe69c4742ab9024382459e583d8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.284275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1333/2680] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:57:49 2022] DELETE /identity/v3/users/aa466fe69c4742ab9024382459e583d8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:49.661274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:49.662731 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:49.679366 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ebf97-81da-4cdd-a99a-a35985bbc764 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:49.756967 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:49.758215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:49.758687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:49.759190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] PATH_INFO: `/v3/projects/58cea02f0a9f420489e9e98ce47f74c8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:49.760578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58cea02f0a9f420489e9e98ce47f74c8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:49.762433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:49.816712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58cea02f0a9f420489e9e98ce47f74c8', 'request_id': 'req-5dc669b0-0371-4665-8df5-8847cbc10bb7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.826237 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58cea02f0a9f420489e9e98ce47f74c8', 'request_id': 'req-5dc669b0-0371-4665-8df5-8847cbc10bb7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.833787 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58cea02f0a9f420489e9e98ce47f74c8', 'request_id': 'req-5dc669b0-0371-4665-8df5-8847cbc10bb7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.846557 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] The token cache is being invalidate because project 58cea02f0a9f420489e9e98ce47f74c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:57:49.846773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.847432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dc669b0-0371-4665-8df5-8847cbc10bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:49.849387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1348/2681] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:57:49 2022] DELETE /identity/v3/projects/58cea02f0a9f420489e9e98ce47f74c8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:49.855639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11cd7-548d-4ce7-94a9-2a2b1b65e54d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:49.856230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11cd7-548d-4ce7-94a9-2a2b1b65e54d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:49.856230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11cd7-548d-4ce7-94a9-2a2b1b65e54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:49.875814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bea11cd7-548d-4ce7-94a9-2a2b1b65e54d None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:49.877134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bea11cd7-548d-4ce7-94a9-2a2b1b65e54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:49.942107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1334/2682] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:49.950978 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8238e3-6700-496b-a315-ad3157c8638b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:49.951632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8238e3-6700-496b-a315-ad3157c8638b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:49.951632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8238e3-6700-496b-a315-ad3157c8638b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:49.982424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8e8238e3-6700-496b-a315-ad3157c8638b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:49.983743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8e8238e3-6700-496b-a315-ad3157c8638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:49.991982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1349/2683] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:49.996442 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea11cd7-548d-4ce7-94a9-2a2b1b65e54d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.008912 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea11cd7-548d-4ce7-94a9-2a2b1b65e54d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.014369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206a21c-f548-4e2b-a650-4a7ccb6c50d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.015031 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206a21c-f548-4e2b-a650-4a7ccb6c50d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.015305 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206a21c-f548-4e2b-a650-4a7ccb6c50d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.015571 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206a21c-f548-4e2b-a650-4a7ccb6c50d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.016739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206a21c-f548-4e2b-a650-4a7ccb6c50d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:57:50.016739 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206a21c-f548-4e2b-a650-4a7ccb6c50d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.018340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206a21c-f548-4e2b-a650-4a7ccb6c50d5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.034072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1335/2684] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:57:50.035034 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8238e3-6700-496b-a315-ad3157c8638b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.044971 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8238e3-6700-496b-a315-ad3157c8638b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.049811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef27ee-52d7-4c65-9609-48a6fd9777e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.050483 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef27ee-52d7-4c65-9609-48a6fd9777e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.050766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef27ee-52d7-4c65-9609-48a6fd9777e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.051042 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef27ee-52d7-4c65-9609-48a6fd9777e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.051754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef27ee-52d7-4c65-9609-48a6fd9777e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.053695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef27ee-52d7-4c65-9609-48a6fd9777e1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.085011 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1350/2685] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:50 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:57:50.087043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206a21c-f548-4e2b-a650-4a7ccb6c50d5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.098089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206a21c-f548-4e2b-a650-4a7ccb6c50d5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.164321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfc4cb-54c3-4bc7-b824-b5a06b3a9bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.165072 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc4cb-54c3-4bc7-b824-b5a06b3a9bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.165564 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc4cb-54c3-4bc7-b824-b5a06b3a9bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.165974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc4cb-54c3-4bc7-b824-b5a06b3a9bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.167206 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfc4cb-54c3-4bc7-b824-b5a06b3a9bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.170547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfc4cb-54c3-4bc7-b824-b5a06b3a9bd1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.195511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1336/2686] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:50 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:50.199463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef27ee-52d7-4c65-9609-48a6fd9777e1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.212728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef27ee-52d7-4c65-9609-48a6fd9777e1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.217801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de26212-f156-41c8-932a-37cc005cbfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.218475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de26212-f156-41c8-932a-37cc005cbfbd admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.218755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de26212-f156-41c8-932a-37cc005cbfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.219023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de26212-f156-41c8-932a-37cc005cbfbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.219743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de26212-f156-41c8-932a-37cc005cbfbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.221832 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de26212-f156-41c8-932a-37cc005cbfbd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.238854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1351/2687] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:50.240799 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfc4cb-54c3-4bc7-b824-b5a06b3a9bd1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.251782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfc4cb-54c3-4bc7-b824-b5a06b3a9bd1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.256775 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaf4f5-34cf-4802-a1db-8e0dc39a5724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.257466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaf4f5-34cf-4802-a1db-8e0dc39a5724 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.257747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaf4f5-34cf-4802-a1db-8e0dc39a5724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.258016 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaf4f5-34cf-4802-a1db-8e0dc39a5724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.258869 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eaf4f5-34cf-4802-a1db-8e0dc39a5724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.260472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eaf4f5-34cf-4802-a1db-8e0dc39a5724 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.272018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1337/2688] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:57:50.277232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de26212-f156-41c8-932a-37cc005cbfbd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.288155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de26212-f156-41c8-932a-37cc005cbfbd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.296022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b733a-955f-4c16-bd4c-38e47ac3b4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.297140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b733a-955f-4c16-bd4c-38e47ac3b4f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.297423 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b733a-955f-4c16-bd4c-38e47ac3b4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.297692 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b733a-955f-4c16-bd4c-38e47ac3b4f9 admin admin] PATH_INFO: `/v3/projects/3d6e3c82dbaa4e1382866677f91f6faf/users/bdf9684a61b14c5f9a6b8b24bb93949b/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.318472 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b733a-955f-4c16-bd4c-38e47ac3b4f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6e3c82dbaa4e1382866677f91f6faf, user_id=bdf9684a61b14c5f9a6b8b24bb93949b, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.321435 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b733a-955f-4c16-bd4c-38e47ac3b4f9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.337338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1352/2689] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:50 2022] PUT /identity/v3/projects/3d6e3c82dbaa4e1382866677f91f6faf/users/bdf9684a61b14c5f9a6b8b24bb93949b/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:50.354925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd7ba8-28d3-4dba-ac09-a83c2c105b94 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.355216 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd7ba8-28d3-4dba-ac09-a83c2c105b94 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.355494 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd7ba8-28d3-4dba-ac09-a83c2c105b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.393925 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a0bd7ba8-28d3-4dba-ac09-a83c2c105b94 None None] MFA Rules not processed for user `bdf9684a61b14c5f9a6b8b24bb93949b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:50.395343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a0bd7ba8-28d3-4dba-ac09-a83c2c105b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:50.471293 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1338/2690] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:50.476745 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b733a-955f-4c16-bd4c-38e47ac3b4f9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.550217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b733a-955f-4c16-bd4c-38e47ac3b4f9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.555985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94016b0d-3706-4ef7-ab09-bb70ce054b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.556711 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94016b0d-3706-4ef7-ab09-bb70ce054b83 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.557000 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94016b0d-3706-4ef7-ab09-bb70ce054b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.557275 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94016b0d-3706-4ef7-ab09-bb70ce054b83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.557979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94016b0d-3706-4ef7-ab09-bb70ce054b83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.560029 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94016b0d-3706-4ef7-ab09-bb70ce054b83 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.593196 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1353/2691] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:50 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:57:50.597398 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd7ba8-28d3-4dba-ac09-a83c2c105b94 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.610557 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd7ba8-28d3-4dba-ac09-a83c2c105b94 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.681970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a182-9cca-4711-8c74-620678bf25d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.682677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a182-9cca-4711-8c74-620678bf25d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.682956 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a182-9cca-4711-8c74-620678bf25d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.683223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a182-9cca-4711-8c74-620678bf25d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.683930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945a182-9cca-4711-8c74-620678bf25d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.687915 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945a182-9cca-4711-8c74-620678bf25d7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.712589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1339/2692] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:50 2022] POST /identity/v3/users => generated 301 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:50.714105 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94016b0d-3706-4ef7-ab09-bb70ce054b83 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.726854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94016b0d-3706-4ef7-ab09-bb70ce054b83 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.732107 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d62c3-2206-4d0e-846e-2b55cc7070df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.733159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d62c3-2206-4d0e-846e-2b55cc7070df admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.733534 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d62c3-2206-4d0e-846e-2b55cc7070df admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.733874 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d62c3-2206-4d0e-846e-2b55cc7070df admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.734674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d62c3-2206-4d0e-846e-2b55cc7070df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.736340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d62c3-2206-4d0e-846e-2b55cc7070df admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.758394 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1354/2693] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:57:50.759887 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a182-9cca-4711-8c74-620678bf25d7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.772292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a182-9cca-4711-8c74-620678bf25d7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.780660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c04795-410d-4b4d-8209-62c35be0a750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.782238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c04795-410d-4b4d-8209-62c35be0a750 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.782600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c04795-410d-4b4d-8209-62c35be0a750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.782948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c04795-410d-4b4d-8209-62c35be0a750 admin admin] PATH_INFO: `/v3/projects/1bb1fe91d1494b4e800da1990c52322a/users/9087d3da9f784e4dbe2ee8cb7879b59b/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.800039 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c04795-410d-4b4d-8209-62c35be0a750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb1fe91d1494b4e800da1990c52322a, user_id=9087d3da9f784e4dbe2ee8cb7879b59b, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.802719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c04795-410d-4b4d-8209-62c35be0a750 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.824074 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1340/2694] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:50 2022] PUT /identity/v3/projects/1bb1fe91d1494b4e800da1990c52322a/users/9087d3da9f784e4dbe2ee8cb7879b59b/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:50.827079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d62c3-2206-4d0e-846e-2b55cc7070df admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.839733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d62c3-2206-4d0e-846e-2b55cc7070df admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.903330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743737d-87a5-49d7-82d2-7883a646c346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.904052 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743737d-87a5-49d7-82d2-7883a646c346 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.904324 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743737d-87a5-49d7-82d2-7883a646c346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.904722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743737d-87a5-49d7-82d2-7883a646c346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.905544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743737d-87a5-49d7-82d2-7883a646c346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.907227 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743737d-87a5-49d7-82d2-7883a646c346 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.922079 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1355/2695] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:50.923307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c04795-410d-4b4d-8209-62c35be0a750 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:50.939531 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c04795-410d-4b4d-8209-62c35be0a750 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:50.944677 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e090cf-c768-4572-97c4-3f92ef40b7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:50.945751 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e090cf-c768-4572-97c4-3f92ef40b7bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.946087 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e090cf-c768-4572-97c4-3f92ef40b7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.946420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e090cf-c768-4572-97c4-3f92ef40b7bf admin admin] PATH_INFO: `/v3/projects/1bb1fe91d1494b4e800da1990c52322a/users/9087d3da9f784e4dbe2ee8cb7879b59b/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:50.948457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e090cf-c768-4572-97c4-3f92ef40b7bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb1fe91d1494b4e800da1990c52322a, user_id=9087d3da9f784e4dbe2ee8cb7879b59b, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:50.951321 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e090cf-c768-4572-97c4-3f92ef40b7bf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:50.963112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1341/2696] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:50 2022] PUT /identity/v3/projects/1bb1fe91d1494b4e800da1990c52322a/users/9087d3da9f784e4dbe2ee8cb7879b59b/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:50.975023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034155ec-daf1-4a7c-97af-c1af68d60d30 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:50.975546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034155ec-daf1-4a7c-97af-c1af68d60d30 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:50.975685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034155ec-daf1-4a7c-97af-c1af68d60d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:51.007039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-034155ec-daf1-4a7c-97af-c1af68d60d30 None None] MFA Rules not processed for user `9087d3da9f784e4dbe2ee8cb7879b59b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:51.008981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-034155ec-daf1-4a7c-97af-c1af68d60d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:51.088002 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1356/2697] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:51.099830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e090cf-c768-4572-97c4-3f92ef40b7bf admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:51.101582 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-19e090cf-c768-4572-97c4-3f92ef40b7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:51.165050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e090cf-c768-4572-97c4-3f92ef40b7bf admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:51.168764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580b58e-1ef0-4943-ab21-6f4a1241d95b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:51.169443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580b58e-1ef0-4943-ab21-6f4a1241d95b service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:51.169751 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580b58e-1ef0-4943-ab21-6f4a1241d95b service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:51.170139 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580b58e-1ef0-4943-ab21-6f4a1241d95b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:51.179939 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580b58e-1ef0-4943-ab21-6f4a1241d95b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:51.181438 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580b58e-1ef0-4943-ab21-6f4a1241d95b service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:51.192128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1342/2698] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:51.350442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034155ec-daf1-4a7c-97af-c1af68d60d30 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:51.426256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034155ec-daf1-4a7c-97af-c1af68d60d30 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:51.432348 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:51.433431 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:51.434076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:51.434360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] PATH_INFO: `/v3/users/bdf9684a61b14c5f9a6b8b24bb93949b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:51.436121 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdf9684a61b14c5f9a6b8b24bb93949b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:51.439198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:51.491372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdf9684a61b14c5f9a6b8b24bb93949b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:51.492191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdf9684a61b14c5f9a6b8b24bb93949b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:51.504871 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdf9684a61b14c5f9a6b8b24bb93949b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:51.513671 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdf9684a61b14c5f9a6b8b24bb93949b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:51.521044 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdf9684a61b14c5f9a6b8b24bb93949b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:51.527363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1357/2699] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:57:51 2022] DELETE /identity/v3/users/bdf9684a61b14c5f9a6b8b24bb93949b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:51.532538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bdfe3-31c6-4829-ad78-d494e33fcad8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:51.533122 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bdfe3-31c6-4829-ad78-d494e33fcad8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:51.533559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bdfe3-31c6-4829-ad78-d494e33fcad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:51.553325 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-876bdfe3-31c6-4829-ad78-d494e33fcad8 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:51.554661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-876bdfe3-31c6-4829-ad78-d494e33fcad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:51.618843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1343/2700] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:51.630439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:51.631918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:51.694118 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b886f0-7065-488b-8a7c-77d7bf65dc70 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:51.698244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232e2c5-c175-4c33-86c0-3941218a7086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:51.698883 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e2c5-c175-4c33-86c0-3941218a7086 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:51.699259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e2c5-c175-4c33-86c0-3941218a7086 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:51.699554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e2c5-c175-4c33-86c0-3941218a7086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:51.709593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232e2c5-c175-4c33-86c0-3941218a7086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:51.711102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232e2c5-c175-4c33-86c0-3941218a7086 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:51.721249 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1358/2701] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:52.045707 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bdfe3-31c6-4829-ad78-d494e33fcad8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:52.047902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-876bdfe3-31c6-4829-ad78-d494e33fcad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:52.066437 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bdfe3-31c6-4829-ad78-d494e33fcad8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:52.075715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:52.077816 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:52.078079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:52.078620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] PATH_INFO: `/v3/projects/3d6e3c82dbaa4e1382866677f91f6faf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:52.081038 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d6e3c82dbaa4e1382866677f91f6faf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:52.084233 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:52.141005 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d6e3c82dbaa4e1382866677f91f6faf', 'request_id': 'req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.153815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d6e3c82dbaa4e1382866677f91f6faf', 'request_id': 'req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.162327 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d6e3c82dbaa4e1382866677f91f6faf', 'request_id': 'req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.173930 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] The token cache is being invalidate because project 3d6e3c82dbaa4e1382866677f91f6faf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:57:52.174230 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.174947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.176617 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1344/2702] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:57:52 2022] DELETE /identity/v3/projects/3d6e3c82dbaa4e1382866677f91f6faf => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:57:52.181191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232e2c5-c175-4c33-86c0-3941218a7086 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:52.192976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232e2c5-c175-4c33-86c0-3941218a7086 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:52.254776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:52.255719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:52.256007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:52.256431 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] PATH_INFO: `/v3/users/9087d3da9f784e4dbe2ee8cb7879b59b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:52.257702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9087d3da9f784e4dbe2ee8cb7879b59b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:52.259678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:52.319242 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9087d3da9f784e4dbe2ee8cb7879b59b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.320436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9087d3da9f784e4dbe2ee8cb7879b59b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.333610 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9087d3da9f784e4dbe2ee8cb7879b59b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.343018 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9087d3da9f784e4dbe2ee8cb7879b59b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.350838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aee8f0ea-82f0-4ba6-ae65-8725414a310e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9087d3da9f784e4dbe2ee8cb7879b59b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.357719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1359/2703] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:57:52 2022] DELETE /identity/v3/users/9087d3da9f784e4dbe2ee8cb7879b59b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:52.683780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:52.685481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:52.705688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02ec1c-2b4c-43a6-b3f5-04b81b49df07 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:52.781950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:52.782760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:52.783020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:52.783287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] PATH_INFO: `/v3/projects/1bb1fe91d1494b4e800da1990c52322a` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:52.784483 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb1fe91d1494b4e800da1990c52322a)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:52.786545 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:52.850875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb1fe91d1494b4e800da1990c52322a', 'request_id': 'req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.861907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb1fe91d1494b4e800da1990c52322a', 'request_id': 'req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.874706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb1fe91d1494b4e800da1990c52322a', 'request_id': 'req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.887054 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] The token cache is being invalidate because project 1bb1fe91d1494b4e800da1990c52322a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:57:52.887465 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.888891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-81635d64-c99f-4fbd-af6e-0dcdb3dcc17c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:57:52.890901 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1345/2704] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:57:52 2022] DELETE /identity/v3/projects/1bb1fe91d1494b4e800da1990c52322a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:52.896258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364ce0e-4907-48e3-92cf-b62c7cdb7bac None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:52.896659 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364ce0e-4907-48e3-92cf-b62c7cdb7bac None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:52.897023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364ce0e-4907-48e3-92cf-b62c7cdb7bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:52.916286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7364ce0e-4907-48e3-92cf-b62c7cdb7bac None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:52.917532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7364ce0e-4907-48e3-92cf-b62c7cdb7bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:52.991558 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1360/2705] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:57:53.000126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80601ab5-d07a-4de9-ac63-f83832d4f2c7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:53.000417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80601ab5-d07a-4de9-ac63-f83832d4f2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:53.000785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80601ab5-d07a-4de9-ac63-f83832d4f2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:53.021740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-80601ab5-d07a-4de9-ac63-f83832d4f2c7 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:53.022944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-80601ab5-d07a-4de9-ac63-f83832d4f2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:53.032970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1346/2706] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:53.037377 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364ce0e-4907-48e3-92cf-b62c7cdb7bac None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:53.048938 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364ce0e-4907-48e3-92cf-b62c7cdb7bac None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:53.054376 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb1e88-2f04-4bb8-aef7-56a1d89d65f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:53.055195 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb1e88-2f04-4bb8-aef7-56a1d89d65f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:53.055445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb1e88-2f04-4bb8-aef7-56a1d89d65f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:53.055809 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb1e88-2f04-4bb8-aef7-56a1d89d65f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:53.056686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb1e88-2f04-4bb8-aef7-56a1d89d65f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:57:53.057060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb1e88-2f04-4bb8-aef7-56a1d89d65f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:53.058856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb1e88-2f04-4bb8-aef7-56a1d89d65f3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:53.072432 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1361/2707] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:57:53.077142 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80601ab5-d07a-4de9-ac63-f83832d4f2c7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:53.086848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80601ab5-d07a-4de9-ac63-f83832d4f2c7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:53.091678 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e438a51-9e66-4180-9410-26e31b4614f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:53.092343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e438a51-9e66-4180-9410-26e31b4614f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:53.092659 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e438a51-9e66-4180-9410-26e31b4614f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:53.092931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e438a51-9e66-4180-9410-26e31b4614f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:53.093803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e438a51-9e66-4180-9410-26e31b4614f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:53.095740 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e438a51-9e66-4180-9410-26e31b4614f2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:53.126947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1347/2708] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:53 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:53.131323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb1e88-2f04-4bb8-aef7-56a1d89d65f3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:53.142627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb1e88-2f04-4bb8-aef7-56a1d89d65f3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:53.210109 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52104b89-770a-4eeb-84e1-19015d2e89d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:53.211210 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52104b89-770a-4eeb-84e1-19015d2e89d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:53.211485 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52104b89-770a-4eeb-84e1-19015d2e89d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:53.211870 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52104b89-770a-4eeb-84e1-19015d2e89d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:53.212597 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52104b89-770a-4eeb-84e1-19015d2e89d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:53.214510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52104b89-770a-4eeb-84e1-19015d2e89d6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:53.238906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1362/2709] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:53 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:53.244024 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e438a51-9e66-4180-9410-26e31b4614f2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:53.255800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e438a51-9e66-4180-9410-26e31b4614f2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:53.260856 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7cfb2-3221-4bc3-9bfb-153733a9f5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:53.261533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7cfb2-3221-4bc3-9bfb-153733a9f5ca admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:53.261824 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7cfb2-3221-4bc3-9bfb-153733a9f5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:53.262085 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7cfb2-3221-4bc3-9bfb-153733a9f5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:53.262885 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7cfb2-3221-4bc3-9bfb-153733a9f5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:53.264598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7cfb2-3221-4bc3-9bfb-153733a9f5ca admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:53.275609 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1348/2710] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:53.279956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52104b89-770a-4eeb-84e1-19015d2e89d6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:53.291849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52104b89-770a-4eeb-84e1-19015d2e89d6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:53.297765 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7fedb-1c6f-4183-9c7b-d8bacaa7b067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:53.298630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7fedb-1c6f-4183-9c7b-d8bacaa7b067 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:53.299232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7fedb-1c6f-4183-9c7b-d8bacaa7b067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:53.299652 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7fedb-1c6f-4183-9c7b-d8bacaa7b067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:53.300563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7fedb-1c6f-4183-9c7b-d8bacaa7b067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:53.302312 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7fedb-1c6f-4183-9c7b-d8bacaa7b067 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:53.315320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1363/2711] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:57:53.319543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7cfb2-3221-4bc3-9bfb-153733a9f5ca admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:53.330099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7cfb2-3221-4bc3-9bfb-153733a9f5ca admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:53.337044 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9177a0-5706-4497-a8e3-0aaa41052f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:53.338416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9177a0-5706-4497-a8e3-0aaa41052f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:53.338990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9177a0-5706-4497-a8e3-0aaa41052f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:53.339380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9177a0-5706-4497-a8e3-0aaa41052f90 admin admin] PATH_INFO: `/v3/projects/916d0d91ea4346ccbd426cb18e329903/users/d9e57208abc44ecc879e2e8a8b90eb84/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:53.357507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9177a0-5706-4497-a8e3-0aaa41052f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=916d0d91ea4346ccbd426cb18e329903, user_id=d9e57208abc44ecc879e2e8a8b90eb84, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:53.361004 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9177a0-5706-4497-a8e3-0aaa41052f90 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:53.376237 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1349/2712] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:53 2022] PUT /identity/v3/projects/916d0d91ea4346ccbd426cb18e329903/users/d9e57208abc44ecc879e2e8a8b90eb84/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:53.381778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c63e7f-5cf0-4adb-9b75-5d0473da16e3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:53.382114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c63e7f-5cf0-4adb-9b75-5d0473da16e3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:53.382499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c63e7f-5cf0-4adb-9b75-5d0473da16e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:53.406201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-51c63e7f-5cf0-4adb-9b75-5d0473da16e3 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:53.407458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-51c63e7f-5cf0-4adb-9b75-5d0473da16e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:53.479099 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1364/2713] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:57:53.494436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9177a0-5706-4497-a8e3-0aaa41052f90 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:53.496352 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5f9177a0-5706-4497-a8e3-0aaa41052f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:53.561148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9177a0-5706-4497-a8e3-0aaa41052f90 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:53.565532 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd09312-0327-497b-b5e2-c7f317a31e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:53.566273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd09312-0327-497b-b5e2-c7f317a31e40 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:53.566616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd09312-0327-497b-b5e2-c7f317a31e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:53.566914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd09312-0327-497b-b5e2-c7f317a31e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:53.576937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd09312-0327-497b-b5e2-c7f317a31e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:53.578507 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd09312-0327-497b-b5e2-c7f317a31e40 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:53.589092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1350/2714] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:54.306875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eea0d4-a259-473e-bf06-4593d456c494 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:54.307796 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eea0d4-a259-473e-bf06-4593d456c494 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:54.308251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eea0d4-a259-473e-bf06-4593d456c494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:54.343869 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c9eea0d4-a259-473e-bf06-4593d456c494 None None] MFA Rules not processed for user `d9e57208abc44ecc879e2e8a8b90eb84`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:54.345535 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c9eea0d4-a259-473e-bf06-4593d456c494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:54.452282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1365/2715] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:54.456688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd09312-0327-497b-b5e2-c7f317a31e40 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:54.481420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd09312-0327-497b-b5e2-c7f317a31e40 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:54.487348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bf06a-ce06-4795-9434-53fb9b04488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:54.487857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bf06a-ce06-4795-9434-53fb9b04488e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:54.488145 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bf06a-ce06-4795-9434-53fb9b04488e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:54.488420 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bf06a-ce06-4795-9434-53fb9b04488e admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:54.489176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bf06a-ce06-4795-9434-53fb9b04488e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:54.491103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bf06a-ce06-4795-9434-53fb9b04488e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:54.525567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1351/2716] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:57:54 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:57:54.529447 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eea0d4-a259-473e-bf06-4593d456c494 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:54.545084 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eea0d4-a259-473e-bf06-4593d456c494 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:54.620539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da4acc-8cd6-4155-9365-79fe751dd5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:54.620539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da4acc-8cd6-4155-9365-79fe751dd5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:54.620539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da4acc-8cd6-4155-9365-79fe751dd5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:54.620539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da4acc-8cd6-4155-9365-79fe751dd5e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:54.621067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da4acc-8cd6-4155-9365-79fe751dd5e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:54.622019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da4acc-8cd6-4155-9365-79fe751dd5e2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:54.645440 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1366/2717] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:57:54 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:57:54.649786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bf06a-ce06-4795-9434-53fb9b04488e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:54.662258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bf06a-ce06-4795-9434-53fb9b04488e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:54.667499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cdd70-7823-49c5-9ce0-c07e30d5a50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:54.668269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cdd70-7823-49c5-9ce0-c07e30d5a50e admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:54.669290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cdd70-7823-49c5-9ce0-c07e30d5a50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:54.669290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cdd70-7823-49c5-9ce0-c07e30d5a50e admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:54.669746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cdd70-7823-49c5-9ce0-c07e30d5a50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:54.671360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cdd70-7823-49c5-9ce0-c07e30d5a50e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:54.697010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1352/2718] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:54.702372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da4acc-8cd6-4155-9365-79fe751dd5e2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:54.716430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da4acc-8cd6-4155-9365-79fe751dd5e2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:54.722248 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a9902-acc6-4bba-bb8f-0806a480ee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:54.723394 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a9902-acc6-4bba-bb8f-0806a480ee9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:54.723762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a9902-acc6-4bba-bb8f-0806a480ee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:54.724134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a9902-acc6-4bba-bb8f-0806a480ee9a admin admin] PATH_INFO: `/v3/projects/83fad07a99e848e4b980996d8b1bf880/users/8fa4ae85a2ba464e9cc7252e74c03415/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:54.744816 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a9902-acc6-4bba-bb8f-0806a480ee9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fad07a99e848e4b980996d8b1bf880, user_id=8fa4ae85a2ba464e9cc7252e74c03415, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:54.747611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a9902-acc6-4bba-bb8f-0806a480ee9a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:54.763511 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1367/2719] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:54 2022] PUT /identity/v3/projects/83fad07a99e848e4b980996d8b1bf880/users/8fa4ae85a2ba464e9cc7252e74c03415/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:54.768279 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cdd70-7823-49c5-9ce0-c07e30d5a50e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:54.782512 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cdd70-7823-49c5-9ce0-c07e30d5a50e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:54.854068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5930623-449c-450b-9629-49f82daebba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:54.854863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5930623-449c-450b-9629-49f82daebba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:54.855299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5930623-449c-450b-9629-49f82daebba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:54.855683 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5930623-449c-450b-9629-49f82daebba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:54.856759 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5930623-449c-450b-9629-49f82daebba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:54.858906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5930623-449c-450b-9629-49f82daebba9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:54.874811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1353/2720] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:57:54.876141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a9902-acc6-4bba-bb8f-0806a480ee9a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:54.888174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a9902-acc6-4bba-bb8f-0806a480ee9a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:54.893593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa89d0c-0edf-48de-9b99-54b4ae1a5690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:54.894696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa89d0c-0edf-48de-9b99-54b4ae1a5690 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:54.895060 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa89d0c-0edf-48de-9b99-54b4ae1a5690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:54.895375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa89d0c-0edf-48de-9b99-54b4ae1a5690 admin admin] PATH_INFO: `/v3/projects/83fad07a99e848e4b980996d8b1bf880/users/8fa4ae85a2ba464e9cc7252e74c03415/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:54.898037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa89d0c-0edf-48de-9b99-54b4ae1a5690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fad07a99e848e4b980996d8b1bf880, user_id=8fa4ae85a2ba464e9cc7252e74c03415, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:54.901053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa89d0c-0edf-48de-9b99-54b4ae1a5690 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:54.913539 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1368/2721] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:57:54 2022] PUT /identity/v3/projects/83fad07a99e848e4b980996d8b1bf880/users/8fa4ae85a2ba464e9cc7252e74c03415/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:57:55.689260 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d194e-bfad-4db9-bd6b-62e340b72c56 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:55.690238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d194e-bfad-4db9-bd6b-62e340b72c56 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:55.690689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d194e-bfad-4db9-bd6b-62e340b72c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:55.723849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-167d194e-bfad-4db9-bd6b-62e340b72c56 None None] MFA Rules not processed for user `8fa4ae85a2ba464e9cc7252e74c03415`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:57:55.728895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-167d194e-bfad-4db9-bd6b-62e340b72c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:55.856349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1354/2722] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:57:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:57:55.867325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa89d0c-0edf-48de-9b99-54b4ae1a5690 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:55.869466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6fa89d0c-0edf-48de-9b99-54b4ae1a5690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:55.941917 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa89d0c-0edf-48de-9b99-54b4ae1a5690 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:55.946282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d78e0-1609-427a-99ac-49c5e5abb713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:55.947159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d78e0-1609-427a-99ac-49c5e5abb713 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:55.947632 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d78e0-1609-427a-99ac-49c5e5abb713 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:55.948091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d78e0-1609-427a-99ac-49c5e5abb713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:56.007527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d78e0-1609-427a-99ac-49c5e5abb713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:56.009360 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d78e0-1609-427a-99ac-49c5e5abb713 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:56.018733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1369/2723] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:57:57.423458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d194e-bfad-4db9-bd6b-62e340b72c56 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:57:57.425370 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-167d194e-bfad-4db9-bd6b-62e340b72c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:57:57.442664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d194e-bfad-4db9-bd6b-62e340b72c56 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:57:57.447526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94c79e-d212-4ac3-bb30-3da9bc24cc79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:57:57.448434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94c79e-d212-4ac3-bb30-3da9bc24cc79 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:57:57.448785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94c79e-d212-4ac3-bb30-3da9bc24cc79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:57:57.449456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94c79e-d212-4ac3-bb30-3da9bc24cc79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:57:57.463369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94c79e-d212-4ac3-bb30-3da9bc24cc79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:57:57.465495 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94c79e-d212-4ac3-bb30-3da9bc24cc79 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:57:57.478743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1355/2724] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:57:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:00.754241 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d304b4-c3b2-4b88-8973-8d8a8c091f08 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:00.754744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d304b4-c3b2-4b88-8973-8d8a8c091f08 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:00.754744 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d304b4-c3b2-4b88-8973-8d8a8c091f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:00.772985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-50d304b4-c3b2-4b88-8973-8d8a8c091f08 None None] MFA Rules not processed for user `b7edbbc857d942d685c124c2c6aac09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:00.774190 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-50d304b4-c3b2-4b88-8973-8d8a8c091f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:00.837779 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1370/2725] 10.222.0.50 () {58 vars in 967 bytes} [Fri Apr 29 18:58:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:00.854416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94c79e-d212-4ac3-bb30-3da9bc24cc79 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:00.856076 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2e94c79e-d212-4ac3-bb30-3da9bc24cc79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:00.929207 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94c79e-d212-4ac3-bb30-3da9bc24cc79 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:00.935527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333db81-f17b-47e0-9e27-82ddeca56f92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cf46333cd034f5dbd0b78503de1f8e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:00.935863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333db81-f17b-47e0-9e27-82ddeca56f92 service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:00.936153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333db81-f17b-47e0-9e27-82ddeca56f92 service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:00.936416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333db81-f17b-47e0-9e27-82ddeca56f92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:00.945285 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333db81-f17b-47e0-9e27-82ddeca56f92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:00.946864 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333db81-f17b-47e0-9e27-82ddeca56f92 service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:00.954753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1356/2726] 10.222.0.50 () {60 vars in 1384 bytes} [Fri Apr 29 18:58:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Apr 29 18:58:03.204975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d304b4-c3b2-4b88-8973-8d8a8c091f08 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:03.206560 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-50d304b4-c3b2-4b88-8973-8d8a8c091f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:03.278097 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d304b4-c3b2-4b88-8973-8d8a8c091f08 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:03.284648 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:03.285526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:03.285799 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:03.286067 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] PATH_INFO: `/v3/users/9558de3a3cd84b8fa24fdbe553ca706a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:03.287322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9558de3a3cd84b8fa24fdbe553ca706a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:03.289308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:03.342532 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9558de3a3cd84b8fa24fdbe553ca706a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:03.343437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9558de3a3cd84b8fa24fdbe553ca706a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:03.354818 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9558de3a3cd84b8fa24fdbe553ca706a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:03.363232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9558de3a3cd84b8fa24fdbe553ca706a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:03.369837 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b756d966-31bc-430e-b378-debfce13548b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9558de3a3cd84b8fa24fdbe553ca706a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:03.375946 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1371/2727] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:58:03 2022] DELETE /identity/v3/users/9558de3a3cd84b8fa24fdbe553ca706a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:03.555883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333db81-f17b-47e0-9e27-82ddeca56f92 service swift] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:03.570845 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333db81-f17b-47e0-9e27-82ddeca56f92 service swift] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:03.634844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:03.635718 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:03.635981 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:03.636250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] PATH_INFO: `/v3/projects/004ea4b077a549868292dad9caaa5515` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:03.637469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=004ea4b077a549868292dad9caaa5515)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:03.639387 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:03.687827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004ea4b077a549868292dad9caaa5515', 'request_id': 'req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:03.697193 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004ea4b077a549868292dad9caaa5515', 'request_id': 'req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:03.704601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004ea4b077a549868292dad9caaa5515', 'request_id': 'req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:03.715434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] The token cache is being invalidate because project 004ea4b077a549868292dad9caaa5515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:58:03.715779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:03.716582 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e3ead346-9b21-4c6b-849d-eef0b33a8fe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:03.718320 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1357/2728] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:58:03 2022] DELETE /identity/v3/projects/004ea4b077a549868292dad9caaa5515 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:03.724383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f008-8c47-49be-b190-d6bb475834cc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:03.724678 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f008-8c47-49be-b190-d6bb475834cc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:03.724931 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f008-8c47-49be-b190-d6bb475834cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:03.743269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ffe9f008-8c47-49be-b190-d6bb475834cc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:03.744524 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ffe9f008-8c47-49be-b190-d6bb475834cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:03.804894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1372/2729] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:03.813646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518e358-223d-4d26-a53d-9427387f8b1e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:03.813938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518e358-223d-4d26-a53d-9427387f8b1e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:03.814214 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518e358-223d-4d26-a53d-9427387f8b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:03.832899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6518e358-223d-4d26-a53d-9427387f8b1e None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:03.834190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6518e358-223d-4d26-a53d-9427387f8b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:03.842375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1358/2730] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:03.847411 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9f008-8c47-49be-b190-d6bb475834cc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:03.857355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9f008-8c47-49be-b190-d6bb475834cc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:03.862404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4323ee-fbc2-42f2-b994-702feb705fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:03.863070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4323ee-fbc2-42f2-b994-702feb705fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:03.863349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4323ee-fbc2-42f2-b994-702feb705fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:03.863625 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4323ee-fbc2-42f2-b994-702feb705fd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:03.864350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4323ee-fbc2-42f2-b994-702feb705fd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:58:03.864818 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4323ee-fbc2-42f2-b994-702feb705fd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:03.866417 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4323ee-fbc2-42f2-b994-702feb705fd1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:03.882235 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1373/2731] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Apr 29 18:58:03.883411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518e358-223d-4d26-a53d-9427387f8b1e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:03.896665 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518e358-223d-4d26-a53d-9427387f8b1e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:03.900802 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c797b59-d892-40a7-a045-6578119c2e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:03.901464 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c797b59-d892-40a7-a045-6578119c2e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:03.901741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c797b59-d892-40a7-a045-6578119c2e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:03.902009 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c797b59-d892-40a7-a045-6578119c2e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:03.902736 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c797b59-d892-40a7-a045-6578119c2e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:03.905228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c797b59-d892-40a7-a045-6578119c2e27 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:03.931021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1359/2732] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:58:03 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:58:03.934998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4323ee-fbc2-42f2-b994-702feb705fd1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:03.945722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4323ee-fbc2-42f2-b994-702feb705fd1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:04.004118 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4568df-f99a-476e-94fb-135d1b77067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:04.004752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4568df-f99a-476e-94fb-135d1b77067c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:04.005020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4568df-f99a-476e-94fb-135d1b77067c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:04.005292 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4568df-f99a-476e-94fb-135d1b77067c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:04.006016 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4568df-f99a-476e-94fb-135d1b77067c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:04.007916 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4568df-f99a-476e-94fb-135d1b77067c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:04.028970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1374/2733] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:58:03 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:58:04.033411 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c797b59-d892-40a7-a045-6578119c2e27 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:04.043800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c797b59-d892-40a7-a045-6578119c2e27 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:04.049750 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b80130-7212-4a14-a2fe-44a08e475328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:04.050516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b80130-7212-4a14-a2fe-44a08e475328 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:04.050794 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b80130-7212-4a14-a2fe-44a08e475328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:04.051064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b80130-7212-4a14-a2fe-44a08e475328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:04.051788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b80130-7212-4a14-a2fe-44a08e475328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:04.053441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b80130-7212-4a14-a2fe-44a08e475328 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:04.064319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1360/2734] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:58:04.069753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4568df-f99a-476e-94fb-135d1b77067c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:04.084790 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4568df-f99a-476e-94fb-135d1b77067c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:04.091982 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9908300-234d-4825-9791-c8223265fdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:04.092722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9908300-234d-4825-9791-c8223265fdbb admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:04.093093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9908300-234d-4825-9791-c8223265fdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:04.093414 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9908300-234d-4825-9791-c8223265fdbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:04.094137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9908300-234d-4825-9791-c8223265fdbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:04.095715 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9908300-234d-4825-9791-c8223265fdbb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:04.105439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1375/2735] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:58:04.109562 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b80130-7212-4a14-a2fe-44a08e475328 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:04.120422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b80130-7212-4a14-a2fe-44a08e475328 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:04.126460 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a664489-decf-4f26-8852-b5c35e13f303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:04.127392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a664489-decf-4f26-8852-b5c35e13f303 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:04.127675 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a664489-decf-4f26-8852-b5c35e13f303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:04.127949 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a664489-decf-4f26-8852-b5c35e13f303 admin admin] PATH_INFO: `/v3/projects/8dbf5a284240437dbae3a69fd39a0d5f/users/f0a1bec61bab4eee93738119c04ae01e/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:04.143359 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a664489-decf-4f26-8852-b5c35e13f303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dbf5a284240437dbae3a69fd39a0d5f, user_id=f0a1bec61bab4eee93738119c04ae01e, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:04.146938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a664489-decf-4f26-8852-b5c35e13f303 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:04.162642 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1361/2736] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:58:04 2022] PUT /identity/v3/projects/8dbf5a284240437dbae3a69fd39a0d5f/users/f0a1bec61bab4eee93738119c04ae01e/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:04.231904 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab4939-7deb-4baf-b242-70863d7a7151 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:04.232775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab4939-7deb-4baf-b242-70863d7a7151 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:04.232775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab4939-7deb-4baf-b242-70863d7a7151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:04.259918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b4ab4939-7deb-4baf-b242-70863d7a7151 None None] MFA Rules not processed for user `f0a1bec61bab4eee93738119c04ae01e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:04.261083 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b4ab4939-7deb-4baf-b242-70863d7a7151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:04.334659 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1376/2737] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:04.345703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a664489-decf-4f26-8852-b5c35e13f303 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:04.347620 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4a664489-decf-4f26-8852-b5c35e13f303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:04.405828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a664489-decf-4f26-8852-b5c35e13f303 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:04.410061 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1c9c8-9f50-4b18-9feb-64689d25d4a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:04.410703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c9c8-9f50-4b18-9feb-64689d25d4a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:04.410980 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c9c8-9f50-4b18-9feb-64689d25d4a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:04.411232 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c9c8-9f50-4b18-9feb-64689d25d4a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:04.419618 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1c9c8-9f50-4b18-9feb-64689d25d4a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:04.421162 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1c9c8-9f50-4b18-9feb-64689d25d4a7 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:04.429580 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1362/2738] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:08.807609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab4939-7deb-4baf-b242-70863d7a7151 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:08.809093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b4ab4939-7deb-4baf-b242-70863d7a7151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:08.868521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab4939-7deb-4baf-b242-70863d7a7151 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:08.873798 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be7d72-8f3a-4909-a01e-b24986d6edf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:08.874537 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7d72-8f3a-4909-a01e-b24986d6edf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:08.874890 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7d72-8f3a-4909-a01e-b24986d6edf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:08.875234 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7d72-8f3a-4909-a01e-b24986d6edf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:08.877770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-47be7d72-8f3a-4909-a01e-b24986d6edf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:08.961457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be7d72-8f3a-4909-a01e-b24986d6edf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:08.962998 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be7d72-8f3a-4909-a01e-b24986d6edf8 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:08.982540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1377/2739] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:09.848778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1c9c8-9f50-4b18-9feb-64689d25d4a7 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:09.850119 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c8d1c9c8-9f50-4b18-9feb-64689d25d4a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:09.985132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1c9c8-9f50-4b18-9feb-64689d25d4a7 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:10.014035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb545105-6b98-4081-a383-deceaefea76f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:10.017667 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb545105-6b98-4081-a383-deceaefea76f service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:10.018435 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb545105-6b98-4081-a383-deceaefea76f service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:10.019174 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb545105-6b98-4081-a383-deceaefea76f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:10.021762 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cb545105-6b98-4081-a383-deceaefea76f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:10.044701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb545105-6b98-4081-a383-deceaefea76f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:10.045357 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb545105-6b98-4081-a383-deceaefea76f service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:10.059785 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1363/2740] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:12.769103 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be7d72-8f3a-4909-a01e-b24986d6edf8 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:12.770747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-47be7d72-8f3a-4909-a01e-b24986d6edf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:12.843019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be7d72-8f3a-4909-a01e-b24986d6edf8 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:12.849313 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:12.850228 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:12.850511 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:12.850928 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] PATH_INFO: `/v3/users/d9e57208abc44ecc879e2e8a8b90eb84` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:12.852139 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9e57208abc44ecc879e2e8a8b90eb84)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:12.854076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:12.900119 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9e57208abc44ecc879e2e8a8b90eb84'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:12.900820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9e57208abc44ecc879e2e8a8b90eb84'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:12.912784 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9e57208abc44ecc879e2e8a8b90eb84'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:12.922213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9e57208abc44ecc879e2e8a8b90eb84'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:12.930681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9e57208abc44ecc879e2e8a8b90eb84'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:12.937875 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1378/2741] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:58:12 2022] DELETE /identity/v3/users/d9e57208abc44ecc879e2e8a8b90eb84 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:13.116225 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb545105-6b98-4081-a383-deceaefea76f service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:13.127594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb545105-6b98-4081-a383-deceaefea76f service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:13.193340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:13.194146 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:13.194410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:13.194871 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] PATH_INFO: `/v3/projects/916d0d91ea4346ccbd426cb18e329903` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:13.196252 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] RBAC: Authorizing `identity:delete_project(project_id=916d0d91ea4346ccbd426cb18e329903)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:13.198186 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:13.246219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '916d0d91ea4346ccbd426cb18e329903', 'request_id': 'req-a671a870-1b0b-401d-8513-ce86177165de'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.256900 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '916d0d91ea4346ccbd426cb18e329903', 'request_id': 'req-a671a870-1b0b-401d-8513-ce86177165de'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.263257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '916d0d91ea4346ccbd426cb18e329903', 'request_id': 'req-a671a870-1b0b-401d-8513-ce86177165de'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.273590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] The token cache is being invalidate because project 916d0d91ea4346ccbd426cb18e329903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:58:13.273875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.274527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.276185 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1364/2742] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:58:13 2022] DELETE /identity/v3/projects/916d0d91ea4346ccbd426cb18e329903 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:13.280318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:13.282088 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:13.296076 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66c916-441e-4a78-8fe9-f1d8956397ca admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:13.356316 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:13.357250 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:13.357549 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:13.357811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] PATH_INFO: `/v3/users/8fa4ae85a2ba464e9cc7252e74c03415` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:13.359063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fa4ae85a2ba464e9cc7252e74c03415)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:13.362327 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:13.409304 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fa4ae85a2ba464e9cc7252e74c03415'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.409991 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fa4ae85a2ba464e9cc7252e74c03415'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.425617 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fa4ae85a2ba464e9cc7252e74c03415'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.433972 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fa4ae85a2ba464e9cc7252e74c03415'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.441562 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bfc9e7df-62cf-43fc-8a18-64a9f89be393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fa4ae85a2ba464e9cc7252e74c03415'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.447592 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1379/2743] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:58:13 2022] DELETE /identity/v3/users/8fa4ae85a2ba464e9cc7252e74c03415 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:13.627573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:13.629470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:13.642772 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671a870-1b0b-401d-8513-ce86177165de admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:13.699716 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:13.700558 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:13.700839 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:13.701107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] PATH_INFO: `/v3/projects/83fad07a99e848e4b980996d8b1bf880` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:13.702271 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83fad07a99e848e4b980996d8b1bf880)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:13.704311 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:13.750195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83fad07a99e848e4b980996d8b1bf880', 'request_id': 'req-706df180-5e8e-4d32-ab89-77b3dbe58821'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.767148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83fad07a99e848e4b980996d8b1bf880', 'request_id': 'req-706df180-5e8e-4d32-ab89-77b3dbe58821'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.767148 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83fad07a99e848e4b980996d8b1bf880', 'request_id': 'req-706df180-5e8e-4d32-ab89-77b3dbe58821'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.776927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] The token cache is being invalidate because project 83fad07a99e848e4b980996d8b1bf880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:58:13.777291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.778000 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-706df180-5e8e-4d32-ab89-77b3dbe58821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:13.779875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1365/2744] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:58:13 2022] DELETE /identity/v3/projects/83fad07a99e848e4b980996d8b1bf880 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:13.785738 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e956c5-7c72-4443-9761-00551a3183f5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:13.786272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e956c5-7c72-4443-9761-00551a3183f5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:13.786272 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e956c5-7c72-4443-9761-00551a3183f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:13.805838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f8e956c5-7c72-4443-9761-00551a3183f5 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:13.807037 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f8e956c5-7c72-4443-9761-00551a3183f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:13.866384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1380/2745] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:13.874338 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f528c8-831b-4b25-b3e1-644551dfe3cc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:13.874653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f528c8-831b-4b25-b3e1-644551dfe3cc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:13.874927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f528c8-831b-4b25-b3e1-644551dfe3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:13.894228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-50f528c8-831b-4b25-b3e1-644551dfe3cc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:13.895399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-50f528c8-831b-4b25-b3e1-644551dfe3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:13.903108 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1366/2746] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:13.907372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e956c5-7c72-4443-9761-00551a3183f5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:13.917438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e956c5-7c72-4443-9761-00551a3183f5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:13.922469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48555a-bc79-48bd-b29f-a3b14ff83bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:13.923193 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48555a-bc79-48bd-b29f-a3b14ff83bbe admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:13.923669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48555a-bc79-48bd-b29f-a3b14ff83bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:13.924134 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48555a-bc79-48bd-b29f-a3b14ff83bbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:13.924891 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48555a-bc79-48bd-b29f-a3b14ff83bbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:58:13.925372 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48555a-bc79-48bd-b29f-a3b14ff83bbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:13.926973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48555a-bc79-48bd-b29f-a3b14ff83bbe admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:13.939213 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1381/2747] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:58:13.943825 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f528c8-831b-4b25-b3e1-644551dfe3cc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:13.954177 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f528c8-831b-4b25-b3e1-644551dfe3cc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:13.959590 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15927043-1f1d-4d53-9c2b-9475946460b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:13.960257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15927043-1f1d-4d53-9c2b-9475946460b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:13.960570 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15927043-1f1d-4d53-9c2b-9475946460b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:13.960843 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15927043-1f1d-4d53-9c2b-9475946460b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:13.961551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15927043-1f1d-4d53-9c2b-9475946460b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:13.963512 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15927043-1f1d-4d53-9c2b-9475946460b6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:13.993399 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1367/2748] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:58:13 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:58:13.996868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48555a-bc79-48bd-b29f-a3b14ff83bbe admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:14.007428 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48555a-bc79-48bd-b29f-a3b14ff83bbe admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:14.065929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ea438-f137-4bf7-ad97-d607f6f77024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:14.066576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ea438-f137-4bf7-ad97-d607f6f77024 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:14.066854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ea438-f137-4bf7-ad97-d607f6f77024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:14.067123 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ea438-f137-4bf7-ad97-d607f6f77024 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:14.067810 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ea438-f137-4bf7-ad97-d607f6f77024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:14.069762 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ea438-f137-4bf7-ad97-d607f6f77024 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:14.093735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1382/2749] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:58:13 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 18:58:14.094973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15927043-1f1d-4d53-9c2b-9475946460b6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:14.105383 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15927043-1f1d-4d53-9c2b-9475946460b6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:14.111064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93318b04-1cb2-4aa1-a436-8fe4a5e38ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:14.111733 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93318b04-1cb2-4aa1-a436-8fe4a5e38ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:14.112011 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93318b04-1cb2-4aa1-a436-8fe4a5e38ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:14.112284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93318b04-1cb2-4aa1-a436-8fe4a5e38ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:14.113043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93318b04-1cb2-4aa1-a436-8fe4a5e38ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:14.114657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93318b04-1cb2-4aa1-a436-8fe4a5e38ec8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:14.125210 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1368/2750] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 18:58:14.129596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ea438-f137-4bf7-ad97-d607f6f77024 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:14.140159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ea438-f137-4bf7-ad97-d607f6f77024 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:14.145568 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b1d9f-e965-4264-8702-42f8ff4f72f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:14.146431 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1d9f-e965-4264-8702-42f8ff4f72f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:14.146754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1d9f-e965-4264-8702-42f8ff4f72f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:14.147093 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1d9f-e965-4264-8702-42f8ff4f72f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:14.147954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b1d9f-e965-4264-8702-42f8ff4f72f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:14.150448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b1d9f-e965-4264-8702-42f8ff4f72f2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:14.162061 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1383/2751] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:58:14.166136 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93318b04-1cb2-4aa1-a436-8fe4a5e38ec8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:14.176300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93318b04-1cb2-4aa1-a436-8fe4a5e38ec8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:14.181340 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894c30b-4eee-42e3-8aa5-2b847bbebda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:14.182215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894c30b-4eee-42e3-8aa5-2b847bbebda9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:14.182489 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894c30b-4eee-42e3-8aa5-2b847bbebda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:14.182767 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894c30b-4eee-42e3-8aa5-2b847bbebda9 admin admin] PATH_INFO: `/v3/projects/2bc14cbc9d024779977bd0bf24445df3/users/61ae678688ae4577b8b31a358d167c0c/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:14.199021 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894c30b-4eee-42e3-8aa5-2b847bbebda9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bc14cbc9d024779977bd0bf24445df3, user_id=61ae678688ae4577b8b31a358d167c0c, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:14.201766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894c30b-4eee-42e3-8aa5-2b847bbebda9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:14.217500 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1369/2752] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:58:14 2022] PUT /identity/v3/projects/2bc14cbc9d024779977bd0bf24445df3/users/61ae678688ae4577b8b31a358d167c0c/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:58:14.281391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bc17f-d478-41d0-94c2-227500a8db6f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:14.281639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bc17f-d478-41d0-94c2-227500a8db6f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:14.281947 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bc17f-d478-41d0-94c2-227500a8db6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:14.310202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a77bc17f-d478-41d0-94c2-227500a8db6f None None] MFA Rules not processed for user `61ae678688ae4577b8b31a358d167c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:14.311421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a77bc17f-d478-41d0-94c2-227500a8db6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:14.376823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1384/2753] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:14.392400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894c30b-4eee-42e3-8aa5-2b847bbebda9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:14.394604 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f894c30b-4eee-42e3-8aa5-2b847bbebda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:14.460706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894c30b-4eee-42e3-8aa5-2b847bbebda9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:14.464527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc531f-85a6-41b1-afda-273bb7ca9012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:14.465179 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc531f-85a6-41b1-afda-273bb7ca9012 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:14.465516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc531f-85a6-41b1-afda-273bb7ca9012 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:14.465778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc531f-85a6-41b1-afda-273bb7ca9012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:14.474704 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc531f-85a6-41b1-afda-273bb7ca9012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:14.476068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc531f-85a6-41b1-afda-273bb7ca9012 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:14.485932 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1370/2754] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:14.602396 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bc17f-d478-41d0-94c2-227500a8db6f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:14.603821 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a77bc17f-d478-41d0-94c2-227500a8db6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:14.673038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bc17f-d478-41d0-94c2-227500a8db6f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:14.678612 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:14.679510 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:14.679791 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:14.680614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] PATH_INFO: `/v3/users/f0a1bec61bab4eee93738119c04ae01e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:14.681521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a1bec61bab4eee93738119c04ae01e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:14.683702 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:14.735437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a1bec61bab4eee93738119c04ae01e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:14.736743 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a1bec61bab4eee93738119c04ae01e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:14.749436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a1bec61bab4eee93738119c04ae01e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:14.760789 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a1bec61bab4eee93738119c04ae01e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:14.770051 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a1bec61bab4eee93738119c04ae01e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:14.786775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1385/2755] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:58:14 2022] DELETE /identity/v3/users/f0a1bec61bab4eee93738119c04ae01e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:14.791770 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42472b-2c65-449c-af17-0bd4ab530305 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:14.792301 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42472b-2c65-449c-af17-0bd4ab530305 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:14.793058 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42472b-2c65-449c-af17-0bd4ab530305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:14.824988 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6c42472b-2c65-449c-af17-0bd4ab530305 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:14.826248 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6c42472b-2c65-449c-af17-0bd4ab530305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:14.911209 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1371/2756] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:14.931032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:14.932775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:14.998476 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207feef-48d2-4e44-afdd-e03bf09dd6a5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:15.001534 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c5de2-c921-496f-85e9-95ea785268dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:15.002188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c5de2-c921-496f-85e9-95ea785268dd service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:15.002465 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c5de2-c921-496f-85e9-95ea785268dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:15.002734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c5de2-c921-496f-85e9-95ea785268dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:15.011820 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c5de2-c921-496f-85e9-95ea785268dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:15.013277 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c5de2-c921-496f-85e9-95ea785268dd service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:15.026081 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1386/2757] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:15.289429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42472b-2c65-449c-af17-0bd4ab530305 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:15.290947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6c42472b-2c65-449c-af17-0bd4ab530305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:15.317453 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42472b-2c65-449c-af17-0bd4ab530305 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:15.324627 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:15.325953 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:15.326516 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:15.327653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] PATH_INFO: `/v3/projects/8dbf5a284240437dbae3a69fd39a0d5f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:15.329110 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dbf5a284240437dbae3a69fd39a0d5f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:15.331347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:15.383815 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dbf5a284240437dbae3a69fd39a0d5f', 'request_id': 'req-5cf79f7d-acb7-492d-aa13-9c784027fdd7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:15.391664 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dbf5a284240437dbae3a69fd39a0d5f', 'request_id': 'req-5cf79f7d-acb7-492d-aa13-9c784027fdd7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:15.398133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dbf5a284240437dbae3a69fd39a0d5f', 'request_id': 'req-5cf79f7d-acb7-492d-aa13-9c784027fdd7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:15.407945 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] The token cache is being invalidate because project 8dbf5a284240437dbae3a69fd39a0d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:58:15.408395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:15.409050 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5cf79f7d-acb7-492d-aa13-9c784027fdd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:15.410637 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1372/2758] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:58:15 2022] DELETE /identity/v3/projects/8dbf5a284240437dbae3a69fd39a0d5f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:17.688527 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c5de2-c921-496f-85e9-95ea785268dd service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:17.690191 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-130c5de2-c921-496f-85e9-95ea785268dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:17.704469 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c5de2-c921-496f-85e9-95ea785268dd service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:17.763395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:17.764273 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:17.764589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:17.764860 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] PATH_INFO: `/v3/users/61ae678688ae4577b8b31a358d167c0c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:17.766200 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61ae678688ae4577b8b31a358d167c0c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:17.768973 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:17.819400 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61ae678688ae4577b8b31a358d167c0c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:17.820111 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61ae678688ae4577b8b31a358d167c0c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:17.833026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61ae678688ae4577b8b31a358d167c0c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:17.842438 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61ae678688ae4577b8b31a358d167c0c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:17.849913 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61ae678688ae4577b8b31a358d167c0c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:17.856100 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1387/2759] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:58:17 2022] DELETE /identity/v3/users/61ae678688ae4577b8b31a358d167c0c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:17.860708 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02cc4-0d0a-45ea-977b-4beeff3d0669 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:17.861213 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02cc4-0d0a-45ea-977b-4beeff3d0669 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:17.861404 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02cc4-0d0a-45ea-977b-4beeff3d0669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:17.879809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-60c02cc4-0d0a-45ea-977b-4beeff3d0669 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:17.880946 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-60c02cc4-0d0a-45ea-977b-4beeff3d0669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:17.941796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1373/2760] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:17.954627 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:17.956458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:18.016416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eebfa5-d98e-4f80-bcb5-bc07a15c38a6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:18.021793 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd6d70-5c76-48e6-9e3b-865920f13264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:18.022446 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd6d70-5c76-48e6-9e3b-865920f13264 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.022718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd6d70-5c76-48e6-9e3b-865920f13264 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.022985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd6d70-5c76-48e6-9e3b-865920f13264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.032987 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd6d70-5c76-48e6-9e3b-865920f13264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:18.034734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd6d70-5c76-48e6-9e3b-865920f13264 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:18.044269 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1388/2761] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 18:58:18.326836 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c02cc4-0d0a-45ea-977b-4beeff3d0669 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:18.328460 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-60c02cc4-0d0a-45ea-977b-4beeff3d0669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:18.342760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c02cc4-0d0a-45ea-977b-4beeff3d0669 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:18.348454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:18.351937 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.352228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.352525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] PATH_INFO: `/v3/projects/2bc14cbc9d024779977bd0bf24445df3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.354080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bc14cbc9d024779977bd0bf24445df3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:18.356449 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:18.440043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bc14cbc9d024779977bd0bf24445df3', 'request_id': 'req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:18.448786 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bc14cbc9d024779977bd0bf24445df3', 'request_id': 'req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:18.455862 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bc14cbc9d024779977bd0bf24445df3', 'request_id': 'req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:18.470942 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] The token cache is being invalidate because project 2bc14cbc9d024779977bd0bf24445df3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:58:18.471402 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:18.472215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2e1149f-1d8b-41f8-b2d6-9aebc1e4f86b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:18.473827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1374/2762] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:58:18 2022] DELETE /identity/v3/projects/2bc14cbc9d024779977bd0bf24445df3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:18.479020 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783dc5e-fb0d-40fd-87ad-500df957af55 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.479374 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783dc5e-fb0d-40fd-87ad-500df957af55 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.479716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783dc5e-fb0d-40fd-87ad-500df957af55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.499778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4783dc5e-fb0d-40fd-87ad-500df957af55 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:18.501202 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4783dc5e-fb0d-40fd-87ad-500df957af55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:18.561390 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1389/2763] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:58:18.569601 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24965262-26a9-49fc-80a2-206b0eb2328c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.569890 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24965262-26a9-49fc-80a2-206b0eb2328c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.570164 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24965262-26a9-49fc-80a2-206b0eb2328c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.589794 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-24965262-26a9-49fc-80a2-206b0eb2328c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:18.591369 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-24965262-26a9-49fc-80a2-206b0eb2328c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:18.603764 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1375/2764] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:18.608085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783dc5e-fb0d-40fd-87ad-500df957af55 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:18.618914 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783dc5e-fb0d-40fd-87ad-500df957af55 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:18.623903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca5afb-a4f3-4b79-aaeb-a76d152f6983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:18.625840 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca5afb-a4f3-4b79-aaeb-a76d152f6983 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.626123 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca5afb-a4f3-4b79-aaeb-a76d152f6983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.626393 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca5afb-a4f3-4b79-aaeb-a76d152f6983 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.627960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca5afb-a4f3-4b79-aaeb-a76d152f6983 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 18:58:18.628445 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca5afb-a4f3-4b79-aaeb-a76d152f6983 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:18.630283 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca5afb-a4f3-4b79-aaeb-a76d152f6983 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:18.643563 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1390/2765] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 18:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 18:58:18.646999 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24965262-26a9-49fc-80a2-206b0eb2328c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:18.656384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24965262-26a9-49fc-80a2-206b0eb2328c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:18.661461 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ef102-5a49-4a03-b9bc-251a3b5801a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:18.662284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ef102-5a49-4a03-b9bc-251a3b5801a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.662660 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ef102-5a49-4a03-b9bc-251a3b5801a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.663028 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ef102-5a49-4a03-b9bc-251a3b5801a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.663827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ef102-5a49-4a03-b9bc-251a3b5801a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:18.665883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ef102-5a49-4a03-b9bc-251a3b5801a7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:18.696415 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1376/2766] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 18:58:18 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:58:18.700722 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca5afb-a4f3-4b79-aaeb-a76d152f6983 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:18.715300 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca5afb-a4f3-4b79-aaeb-a76d152f6983 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:18.778674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf000f9-159e-4b87-9a39-1a0c57e93619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:18.779350 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf000f9-159e-4b87-9a39-1a0c57e93619 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.779630 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf000f9-159e-4b87-9a39-1a0c57e93619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.780027 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf000f9-159e-4b87-9a39-1a0c57e93619 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.780761 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf000f9-159e-4b87-9a39-1a0c57e93619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:18.782723 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf000f9-159e-4b87-9a39-1a0c57e93619 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:18.804344 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1391/2767] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 18:58:18 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 18:58:18.808614 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ef102-5a49-4a03-b9bc-251a3b5801a7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:18.821542 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ef102-5a49-4a03-b9bc-251a3b5801a7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:18.827630 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1a261-40ac-4113-9165-16c58494e102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:18.828351 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1a261-40ac-4113-9165-16c58494e102 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.828720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1a261-40ac-4113-9165-16c58494e102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.829173 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1a261-40ac-4113-9165-16c58494e102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.829917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1a261-40ac-4113-9165-16c58494e102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:18.831613 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1a261-40ac-4113-9165-16c58494e102 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:18.844195 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1377/2768] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:58:18.848872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf000f9-159e-4b87-9a39-1a0c57e93619 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:18.859499 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf000f9-159e-4b87-9a39-1a0c57e93619 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:18.864808 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58ce7e-1842-471a-8e0e-0cbaa666f412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:18.865478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58ce7e-1842-471a-8e0e-0cbaa666f412 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.865770 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58ce7e-1842-471a-8e0e-0cbaa666f412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.866044 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58ce7e-1842-471a-8e0e-0cbaa666f412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.866799 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58ce7e-1842-471a-8e0e-0cbaa666f412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:18.868442 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58ce7e-1842-471a-8e0e-0cbaa666f412 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:18.878829 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1392/2769] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 18:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 18:58:18.883292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1a261-40ac-4113-9165-16c58494e102 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:18.893699 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1a261-40ac-4113-9165-16c58494e102 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:18.898973 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04ea5d-4c89-4fa4-add2-d8a538aa796a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:18.900106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04ea5d-4c89-4fa4-add2-d8a538aa796a admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.900384 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04ea5d-4c89-4fa4-add2-d8a538aa796a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.900712 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04ea5d-4c89-4fa4-add2-d8a538aa796a admin admin] PATH_INFO: `/v3/projects/2cc8d07d602c48a68f9b13cd4d7f2997/users/f361cb94552a47faa24424978efa7250/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.916332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04ea5d-4c89-4fa4-add2-d8a538aa796a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc8d07d602c48a68f9b13cd4d7f2997, user_id=f361cb94552a47faa24424978efa7250, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:18.919128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04ea5d-4c89-4fa4-add2-d8a538aa796a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:18.935219 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1378/2770] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 18:58:18 2022] PUT /identity/v3/projects/2cc8d07d602c48a68f9b13cd4d7f2997/users/f361cb94552a47faa24424978efa7250/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:18.946068 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fbf95-93ca-4137-bbf3-9c57f5d410cc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:18.946359 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fbf95-93ca-4137-bbf3-9c57f5d410cc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:18.946639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fbf95-93ca-4137-bbf3-9c57f5d410cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:18.974205 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e00fbf95-93ca-4137-bbf3-9c57f5d410cc None None] MFA Rules not processed for user `f361cb94552a47faa24424978efa7250`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:18.975412 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e00fbf95-93ca-4137-bbf3-9c57f5d410cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:19.039330 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1393/2771] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:58:19.053389 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04ea5d-4c89-4fa4-add2-d8a538aa796a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:19.055334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8b04ea5d-4c89-4fa4-add2-d8a538aa796a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:19.156027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04ea5d-4c89-4fa4-add2-d8a538aa796a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:19.160520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060427ff-d73c-49e0-8c92-19b3e5b75705 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6552c41f1c654ef5bfdd940e3b807dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:19.161228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060427ff-d73c-49e0-8c92-19b3e5b75705 service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:19.161559 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060427ff-d73c-49e0-8c92-19b3e5b75705 service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:19.161782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060427ff-d73c-49e0-8c92-19b3e5b75705 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:19.172309 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060427ff-d73c-49e0-8c92-19b3e5b75705 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:19.173873 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060427ff-d73c-49e0-8c92-19b3e5b75705 service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:19.188342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1379/2772] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:26.270668 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fbf95-93ca-4137-bbf3-9c57f5d410cc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:26.272352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e00fbf95-93ca-4137-bbf3-9c57f5d410cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:26.354429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fbf95-93ca-4137-bbf3-9c57f5d410cc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:26.360317 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d674ee-ac46-4709-b7e6-fc0e0f939285 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:26.361014 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d674ee-ac46-4709-b7e6-fc0e0f939285 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:26.361296 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d674ee-ac46-4709-b7e6-fc0e0f939285 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:26.361578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d674ee-ac46-4709-b7e6-fc0e0f939285 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:26.363433 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3d674ee-ac46-4709-b7e6-fc0e0f939285 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:26.431720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d674ee-ac46-4709-b7e6-fc0e0f939285 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:26.433353 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d674ee-ac46-4709-b7e6-fc0e0f939285 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:26.444609 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1394/2773] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:29.270990 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060427ff-d73c-49e0-8c92-19b3e5b75705 service glance] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:29.272672 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-060427ff-d73c-49e0-8c92-19b3e5b75705 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:29.330947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060427ff-d73c-49e0-8c92-19b3e5b75705 service glance] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:29.335766 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d86dce-9057-4d12-a0d5-8c8133c98a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:29.336428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d86dce-9057-4d12-a0d5-8c8133c98a33 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:29.336737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d86dce-9057-4d12-a0d5-8c8133c98a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:29.337025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d86dce-9057-4d12-a0d5-8c8133c98a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:29.338906 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a8d86dce-9057-4d12-a0d5-8c8133c98a33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:29.406245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d86dce-9057-4d12-a0d5-8c8133c98a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:29.408577 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d86dce-9057-4d12-a0d5-8c8133c98a33 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:29.419291 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1380/2774] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:29.866166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e4f7e-149d-469d-98ea-01c485bb2926 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:29.866752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e4f7e-149d-469d-98ea-01c485bb2926 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:29.866752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e4f7e-149d-469d-98ea-01c485bb2926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:29.888157 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fc6e4f7e-149d-469d-98ea-01c485bb2926 None None] MFA Rules not processed for user `cc76dcb94459489f965c9205f2d461cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:58:29.889580 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fc6e4f7e-149d-469d-98ea-01c485bb2926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:29.896930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1395/2775] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:58:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 18:58:29.937798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d86dce-9057-4d12-a0d5-8c8133c98a33 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:29.952972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d86dce-9057-4d12-a0d5-8c8133c98a33 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:29.957395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c56a4-4b44-4260-92d5-fa7103dec6bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6552c41f1c654ef5bfdd940e3b807dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:29.958119 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c56a4-4b44-4260-92d5-fa7103dec6bb service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:29.958481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c56a4-4b44-4260-92d5-fa7103dec6bb service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:29.958788 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c56a4-4b44-4260-92d5-fa7103dec6bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:29.960720 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f91c56a4-4b44-4260-92d5-fa7103dec6bb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:29.975773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c56a4-4b44-4260-92d5-fa7103dec6bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:29.977315 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c56a4-4b44-4260-92d5-fa7103dec6bb service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:29.996673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1381/2776] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:30.912457 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e4f7e-149d-469d-98ea-01c485bb2926 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:30.914384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fc6e4f7e-149d-469d-98ea-01c485bb2926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:30.929910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e4f7e-149d-469d-98ea-01c485bb2926 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:30.933675 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fff3a0-bbc0-4a09-9b00-f15bda76b78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:30.934325 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fff3a0-bbc0-4a09-9b00-f15bda76b78d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:30.934594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fff3a0-bbc0-4a09-9b00-f15bda76b78d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:30.934867 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fff3a0-bbc0-4a09-9b00-f15bda76b78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:30.937032 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-59fff3a0-bbc0-4a09-9b00-f15bda76b78d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:30.953186 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fff3a0-bbc0-4a09-9b00-f15bda76b78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:30.957835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fff3a0-bbc0-4a09-9b00-f15bda76b78d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:30.986725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1396/2777] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:58:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:58:37.109881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c56a4-4b44-4260-92d5-fa7103dec6bb service glance] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:37.111463 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f91c56a4-4b44-4260-92d5-fa7103dec6bb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:37.137791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c56a4-4b44-4260-92d5-fa7103dec6bb service glance] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:37.142943 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:37.143828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:37.144112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:37.144382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] PATH_INFO: `/v3/users/a1569db96d824c0e9f16b386df5e3413` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:37.146454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1569db96d824c0e9f16b386df5e3413)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:37.148402 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:37.196526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1569db96d824c0e9f16b386df5e3413'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.197170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1569db96d824c0e9f16b386df5e3413'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.208543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1569db96d824c0e9f16b386df5e3413'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.215231 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1569db96d824c0e9f16b386df5e3413'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.223064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1569db96d824c0e9f16b386df5e3413'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.229086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1382/2778] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:58:37 2022] DELETE /identity/v3/users/a1569db96d824c0e9f16b386df5e3413 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:37.413728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fff3a0-bbc0-4a09-9b00-f15bda76b78d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:37.424464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fff3a0-bbc0-4a09-9b00-f15bda76b78d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:37.481551 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:37.482418 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:37.482712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:37.482965 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] PATH_INFO: `/v3/projects/8853dc3f41e147018cc50778b57b9c97` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:37.484179 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8853dc3f41e147018cc50778b57b9c97)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:37.486110 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:37.536985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8853dc3f41e147018cc50778b57b9c97', 'request_id': 'req-ac373d52-d6d6-44bf-9885-2621d3811ec8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.545872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8853dc3f41e147018cc50778b57b9c97', 'request_id': 'req-ac373d52-d6d6-44bf-9885-2621d3811ec8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.552906 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8853dc3f41e147018cc50778b57b9c97', 'request_id': 'req-ac373d52-d6d6-44bf-9885-2621d3811ec8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.564659 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] The token cache is being invalidate because project 8853dc3f41e147018cc50778b57b9c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:58:37.564949 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.565683 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.567855 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1397/2779] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:58:37 2022] DELETE /identity/v3/projects/8853dc3f41e147018cc50778b57b9c97 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:37.571719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:37.573266 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:37.586874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e8391-bfbb-4afe-a088-c98d5aa2ec4b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:37.642628 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:37.643512 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:37.643790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:37.644059 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] PATH_INFO: `/v3/users/4648aa4adcfc480b968b9b2abd2d7b17` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:37.657478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4648aa4adcfc480b968b9b2abd2d7b17)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:37.659408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:37.709782 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4648aa4adcfc480b968b9b2abd2d7b17'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.710449 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4648aa4adcfc480b968b9b2abd2d7b17'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.721742 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4648aa4adcfc480b968b9b2abd2d7b17'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.728466 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4648aa4adcfc480b968b9b2abd2d7b17'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.738168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4648aa4adcfc480b968b9b2abd2d7b17'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:37.744581 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1383/2780] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:58:37 2022] DELETE /identity/v3/users/4648aa4adcfc480b968b9b2abd2d7b17 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:58:37.933623 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:58:37.935477 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:58:37.949730 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac373d52-d6d6-44bf-9885-2621d3811ec8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:58:38.006896 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:58:38.007764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:58:38.008039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:58:38.008308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] PATH_INFO: `/v3/projects/c8229262ea9a40e4970b89e8adadc884` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:58:38.018364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8229262ea9a40e4970b89e8adadc884)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:58:38.020455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:58:38.085557 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8229262ea9a40e4970b89e8adadc884', 'request_id': 'req-840a6857-53c1-4816-9881-d75770f2a7a7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:38.095008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8229262ea9a40e4970b89e8adadc884', 'request_id': 'req-840a6857-53c1-4816-9881-d75770f2a7a7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:38.105004 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8229262ea9a40e4970b89e8adadc884', 'request_id': 'req-840a6857-53c1-4816-9881-d75770f2a7a7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:38.119039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] The token cache is being invalidate because project c8229262ea9a40e4970b89e8adadc884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:58:38.119323 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:38.120033 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-840a6857-53c1-4816-9881-d75770f2a7a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:58:38.121719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1398/2781] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:58:37 2022] DELETE /identity/v3/projects/c8229262ea9a40e4970b89e8adadc884 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 18:59:10.593253 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:59:10.594934 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:59:10.608400 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2403d92-e956-4a75-9d03-38895160da2c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:59:10.667861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:59:10.668979 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:59:10.669307 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:59:10.669592 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] PATH_INFO: `/v3/users/f361cb94552a47faa24424978efa7250` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:59:10.671010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f361cb94552a47faa24424978efa7250)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:59:10.673015 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:59:10.723204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f361cb94552a47faa24424978efa7250'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:59:10.723868 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f361cb94552a47faa24424978efa7250'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:59:10.735168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f361cb94552a47faa24424978efa7250'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:59:10.741803 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f361cb94552a47faa24424978efa7250'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:59:10.751273 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f361cb94552a47faa24424978efa7250'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:59:10.757147 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1384/2782] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 18:59:10 2022] DELETE /identity/v3/users/f361cb94552a47faa24424978efa7250 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:59:10.762897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b5fd2-ee6b-4705-b39c-8a16b917395c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:59:10.763769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b5fd2-ee6b-4705-b39c-8a16b917395c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:59:10.763769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b5fd2-ee6b-4705-b39c-8a16b917395c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:59:10.784682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7a3b5fd2-ee6b-4705-b39c-8a16b917395c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 18:59:10.785870 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7a3b5fd2-ee6b-4705-b39c-8a16b917395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:59:10.844993 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1399/2783] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 18:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 18:59:10.854776 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:59:10.856300 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:59:10.913343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420c870-7d7d-4a43-a678-acd3fad18834 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:59:10.917391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fad06-593d-43fd-be24-26449bc7c409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:59:10.918405 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fad06-593d-43fd-be24-26449bc7c409 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:59:10.918586 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fad06-593d-43fd-be24-26449bc7c409 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:59:10.918907 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fad06-593d-43fd-be24-26449bc7c409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:59:10.928107 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fad06-593d-43fd-be24-26449bc7c409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:59:10.929576 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fad06-593d-43fd-be24-26449bc7c409 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:59:10.939333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1385/2784] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 18:59:11.212408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b5fd2-ee6b-4705-b39c-8a16b917395c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:59:11.214184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7a3b5fd2-ee6b-4705-b39c-8a16b917395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:59:11.225698 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b5fd2-ee6b-4705-b39c-8a16b917395c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:59:11.230294 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:59:11.231244 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:59:11.231520 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:59:11.231818 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] PATH_INFO: `/v3/projects/2cc8d07d602c48a68f9b13cd4d7f2997` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:59:11.233068 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc8d07d602c48a68f9b13cd4d7f2997)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:59:11.234902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:59:11.284422 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc8d07d602c48a68f9b13cd4d7f2997', 'request_id': 'req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:59:11.293844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc8d07d602c48a68f9b13cd4d7f2997', 'request_id': 'req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:59:11.303096 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc8d07d602c48a68f9b13cd4d7f2997', 'request_id': 'req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:59:11.314555 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] The token cache is being invalidate because project 2cc8d07d602c48a68f9b13cd4d7f2997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 18:59:11.314844 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:59:11.315552 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 18:59:11.317349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1400/2785] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 18:59:11 2022] DELETE /identity/v3/projects/2cc8d07d602c48a68f9b13cd4d7f2997 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 18:59:31.243083 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fad06-593d-43fd-be24-26449bc7c409 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 18:59:31.252380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fad06-593d-43fd-be24-26449bc7c409 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 18:59:31.306713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0189da-20c5-4fbb-975d-971c01f45655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 18:59:31.307362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0189da-20c5-4fbb-975d-971c01f45655 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 18:59:31.307646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0189da-20c5-4fbb-975d-971c01f45655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 18:59:31.307918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0189da-20c5-4fbb-975d-971c01f45655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 18:59:31.310017 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2c0189da-20c5-4fbb-975d-971c01f45655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 18:59:31.320649 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0189da-20c5-4fbb-975d-971c01f45655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 18:59:31.322077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0189da-20c5-4fbb-975d-971c01f45655 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 18:59:31.330020 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1386/2786] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 18:59:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:00:32.705750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:00:32.707596 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:00:32.796058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c436bf-5f51-4614-8c3b-4cb31cf5da1d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:00:32.802823 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed9711-ee8d-4805-8ba7-f8783c512bfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:00:32.803544 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed9711-ee8d-4805-8ba7-f8783c512bfb service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:00:32.803872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed9711-ee8d-4805-8ba7-f8783c512bfb service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:00:32.804150 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed9711-ee8d-4805-8ba7-f8783c512bfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:00:32.806100 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-82ed9711-ee8d-4805-8ba7-f8783c512bfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:00:32.821448 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed9711-ee8d-4805-8ba7-f8783c512bfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:00:32.823070 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed9711-ee8d-4805-8ba7-f8783c512bfb service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:00:32.845308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1401/2787] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:00:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 19:00:36.623165 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7573d92-1c0c-4d13-b742-4c31115a5fec None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:00:36.623741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7573d92-1c0c-4d13-b742-4c31115a5fec None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:00:36.623968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7573d92-1c0c-4d13-b742-4c31115a5fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:00:36.642444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b7573d92-1c0c-4d13-b742-4c31115a5fec None None] MFA Rules not processed for user `cc76dcb94459489f965c9205f2d461cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:00:36.643670 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b7573d92-1c0c-4d13-b742-4c31115a5fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:00:36.706360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1387/2788] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:00:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:00:37.339645 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed9711-ee8d-4805-8ba7-f8783c512bfb service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:00:37.341328 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-82ed9711-ee8d-4805-8ba7-f8783c512bfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:00:37.410380 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed9711-ee8d-4805-8ba7-f8783c512bfb service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:00:37.415203 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6deb7e-8748-433e-9744-86bc00c74878 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e934c8e7dd0425cac68eb91e5e2c0ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:00:37.415850 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6deb7e-8748-433e-9744-86bc00c74878 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:00:37.416129 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6deb7e-8748-433e-9744-86bc00c74878 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:00:37.416397 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6deb7e-8748-433e-9744-86bc00c74878 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:00:37.418171 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9a6deb7e-8748-433e-9744-86bc00c74878 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:00:37.429287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6deb7e-8748-433e-9744-86bc00c74878 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:00:37.430835 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6deb7e-8748-433e-9744-86bc00c74878 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:00:37.440604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1402/2789] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:00:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:00:46.227336 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7573d92-1c0c-4d13-b742-4c31115a5fec None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:00:46.228927 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b7573d92-1c0c-4d13-b742-4c31115a5fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:00:46.247671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7573d92-1c0c-4d13-b742-4c31115a5fec None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:00:46.253485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b36f69-df18-4c0e-a704-b45464ddd641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:00:46.254137 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b36f69-df18-4c0e-a704-b45464ddd641 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:00:46.254424 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b36f69-df18-4c0e-a704-b45464ddd641 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:00:46.254690 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b36f69-df18-4c0e-a704-b45464ddd641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:00:46.256557 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-31b36f69-df18-4c0e-a704-b45464ddd641 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:00:46.269360 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b36f69-df18-4c0e-a704-b45464ddd641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:00:46.270944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b36f69-df18-4c0e-a704-b45464ddd641 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:00:46.282216 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1388/2790] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:00:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:02:20.094464 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6deb7e-8748-433e-9744-86bc00c74878 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:20.106064 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6deb7e-8748-433e-9744-86bc00c74878 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:20.111733 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa25a1e-d529-4bd9-ae30-6cbadad1e081 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e934c8e7dd0425cac68eb91e5e2c0ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:20.112456 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa25a1e-d529-4bd9-ae30-6cbadad1e081 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:20.112769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa25a1e-d529-4bd9-ae30-6cbadad1e081 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:20.113048 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa25a1e-d529-4bd9-ae30-6cbadad1e081 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:20.114801 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bfa25a1e-d529-4bd9-ae30-6cbadad1e081 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:20.126764 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa25a1e-d529-4bd9-ae30-6cbadad1e081 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:20.128361 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa25a1e-d529-4bd9-ae30-6cbadad1e081 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:20.138253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1403/2791] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:02:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 19:02:26.159968 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b36f69-df18-4c0e-a704-b45464ddd641 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:26.161572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-31b36f69-df18-4c0e-a704-b45464ddd641 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:26.174254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b36f69-df18-4c0e-a704-b45464ddd641 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:26.178602 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606dfd-564c-4cd5-b413-0b2fc8ddc506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:26.179246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606dfd-564c-4cd5-b413-0b2fc8ddc506 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:26.179523 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606dfd-564c-4cd5-b413-0b2fc8ddc506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:26.179800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606dfd-564c-4cd5-b413-0b2fc8ddc506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:26.181805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3c606dfd-564c-4cd5-b413-0b2fc8ddc506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:26.255827 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606dfd-564c-4cd5-b413-0b2fc8ddc506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:26.257931 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606dfd-564c-4cd5-b413-0b2fc8ddc506 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:26.272966 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1389/2792] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:02:29.290923 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa25a1e-d529-4bd9-ae30-6cbadad1e081 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:29.292651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bfa25a1e-d529-4bd9-ae30-6cbadad1e081 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:29.309834 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa25a1e-d529-4bd9-ae30-6cbadad1e081 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:29.315013 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:29.315853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:29.316138 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:29.316414 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] PATH_INFO: `/v3/users/cc76dcb94459489f965c9205f2d461cb` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:29.317651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc76dcb94459489f965c9205f2d461cb)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:29.319611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:29.373019 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc76dcb94459489f965c9205f2d461cb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:29.373720 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc76dcb94459489f965c9205f2d461cb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:29.385838 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc76dcb94459489f965c9205f2d461cb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:29.393775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc76dcb94459489f965c9205f2d461cb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:29.401005 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-23dff55f-83cb-4539-90f9-24ca3a0e5e2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc76dcb94459489f965c9205f2d461cb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:29.407286 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1404/2793] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 19:02:29 2022] DELETE /identity/v3/users/cc76dcb94459489f965c9205f2d461cb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:29.585092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606dfd-564c-4cd5-b413-0b2fc8ddc506 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:29.596395 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606dfd-564c-4cd5-b413-0b2fc8ddc506 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:29.653416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:29.654288 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:29.654567 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:29.654837 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] PATH_INFO: `/v3/projects/9da6a94754bf43c4a0119272fb920c06` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:29.656534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9da6a94754bf43c4a0119272fb920c06)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:29.658472 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:29.706481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9da6a94754bf43c4a0119272fb920c06', 'request_id': 'req-8f790724-cc68-412b-ac53-545ed7c54f01'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:29.713841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9da6a94754bf43c4a0119272fb920c06', 'request_id': 'req-8f790724-cc68-412b-ac53-545ed7c54f01'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:29.720641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9da6a94754bf43c4a0119272fb920c06', 'request_id': 'req-8f790724-cc68-412b-ac53-545ed7c54f01'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:29.730481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] The token cache is being invalidate because project 9da6a94754bf43c4a0119272fb920c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 19:02:29.730771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:29.731403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8f790724-cc68-412b-ac53-545ed7c54f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:29.732989 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1390/2794] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 19:02:29 2022] DELETE /identity/v3/projects/9da6a94754bf43c4a0119272fb920c06 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:29.738754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fac03-5283-4f20-b001-462f27eb0774 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:29.739047 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fac03-5283-4f20-b001-462f27eb0774 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:29.739326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fac03-5283-4f20-b001-462f27eb0774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:29.759375 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-616fac03-5283-4f20-b001-462f27eb0774 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:29.760631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-616fac03-5283-4f20-b001-462f27eb0774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:29.820174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1405/2795] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:29.828773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d08dfc-403e-4d61-9bbc-ff211dae96fb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:29.829064 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d08dfc-403e-4d61-9bbc-ff211dae96fb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:29.829343 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d08dfc-403e-4d61-9bbc-ff211dae96fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:29.848092 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c9d08dfc-403e-4d61-9bbc-ff211dae96fb None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:29.849322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c9d08dfc-403e-4d61-9bbc-ff211dae96fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:29.857543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1391/2796] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:29.862421 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fac03-5283-4f20-b001-462f27eb0774 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:29.872318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fac03-5283-4f20-b001-462f27eb0774 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:29.878021 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58704f-0a41-4757-9e0c-4a92eda20cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:29.878685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58704f-0a41-4757-9e0c-4a92eda20cea admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:29.878964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58704f-0a41-4757-9e0c-4a92eda20cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:29.879236 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58704f-0a41-4757-9e0c-4a92eda20cea admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:29.879970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58704f-0a41-4757-9e0c-4a92eda20cea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 19:02:29.880287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58704f-0a41-4757-9e0c-4a92eda20cea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:29.881959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58704f-0a41-4757-9e0c-4a92eda20cea admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:29.894349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1406/2797] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 19:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 19:02:29.898717 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d08dfc-403e-4d61-9bbc-ff211dae96fb None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:29.908944 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d08dfc-403e-4d61-9bbc-ff211dae96fb None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:29.914208 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0954ab-e303-4c5f-a9d6-77e6194fb7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:29.914867 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0954ab-e303-4c5f-a9d6-77e6194fb7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:29.915152 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0954ab-e303-4c5f-a9d6-77e6194fb7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:29.915421 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0954ab-e303-4c5f-a9d6-77e6194fb7d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:29.916103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0954ab-e303-4c5f-a9d6-77e6194fb7d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:29.918027 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0954ab-e303-4c5f-a9d6-77e6194fb7d0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:29.945319 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1392/2798] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 19:02:29 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:02:29.949618 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58704f-0a41-4757-9e0c-4a92eda20cea admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:29.960078 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58704f-0a41-4757-9e0c-4a92eda20cea admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:30.034856 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450085d-c17e-40f1-be9f-5af312b75e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:30.035604 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450085d-c17e-40f1-be9f-5af312b75e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:30.035887 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450085d-c17e-40f1-be9f-5af312b75e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:30.036156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450085d-c17e-40f1-be9f-5af312b75e7b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:30.037054 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450085d-c17e-40f1-be9f-5af312b75e7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:30.040043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450085d-c17e-40f1-be9f-5af312b75e7b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:30.069181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1407/2799] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 19:02:29 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 19:02:30.075289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0954ab-e303-4c5f-a9d6-77e6194fb7d0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:30.086643 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0954ab-e303-4c5f-a9d6-77e6194fb7d0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:30.092935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b3e6e-a1ab-418a-9eba-d2d9c283bc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:30.093619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b3e6e-a1ab-418a-9eba-d2d9c283bc55 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:30.093919 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b3e6e-a1ab-418a-9eba-d2d9c283bc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:30.094168 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b3e6e-a1ab-418a-9eba-d2d9c283bc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:30.094883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b3e6e-a1ab-418a-9eba-d2d9c283bc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:30.096485 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b3e6e-a1ab-418a-9eba-d2d9c283bc55 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:30.108593 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1393/2800] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:02:30.112751 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450085d-c17e-40f1-be9f-5af312b75e7b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:30.122611 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450085d-c17e-40f1-be9f-5af312b75e7b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:30.128251 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dda1ae-21ac-4a62-9e1f-a87bad4db670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:30.128948 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dda1ae-21ac-4a62-9e1f-a87bad4db670 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:30.129225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dda1ae-21ac-4a62-9e1f-a87bad4db670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:30.129492 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dda1ae-21ac-4a62-9e1f-a87bad4db670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:30.130197 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dda1ae-21ac-4a62-9e1f-a87bad4db670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:30.131792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dda1ae-21ac-4a62-9e1f-a87bad4db670 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:30.142670 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1408/2801] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:02:30.146684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b3e6e-a1ab-418a-9eba-d2d9c283bc55 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:30.156995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b3e6e-a1ab-418a-9eba-d2d9c283bc55 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:30.162079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a18a71-af97-4f17-8a3e-74b131a7203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:30.162947 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a18a71-af97-4f17-8a3e-74b131a7203d admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:30.163221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a18a71-af97-4f17-8a3e-74b131a7203d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:30.163492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a18a71-af97-4f17-8a3e-74b131a7203d admin admin] PATH_INFO: `/v3/projects/d0c4f2c188644461881ad70639d5f99c/users/a5428cada65b43afa6df79f4bf748c6e/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:30.178246 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a18a71-af97-4f17-8a3e-74b131a7203d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c4f2c188644461881ad70639d5f99c, user_id=a5428cada65b43afa6df79f4bf748c6e, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:30.180916 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a18a71-af97-4f17-8a3e-74b131a7203d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:30.200594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1394/2802] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:02:30 2022] PUT /identity/v3/projects/d0c4f2c188644461881ad70639d5f99c/users/a5428cada65b43afa6df79f4bf748c6e/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:30.212803 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b8c81-5694-432b-bdc6-7fe0bd977115 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:30.213091 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b8c81-5694-432b-bdc6-7fe0bd977115 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:30.213364 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b8c81-5694-432b-bdc6-7fe0bd977115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:30.241814 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2d8b8c81-5694-432b-bdc6-7fe0bd977115 None None] MFA Rules not processed for user `a5428cada65b43afa6df79f4bf748c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:30.243022 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2d8b8c81-5694-432b-bdc6-7fe0bd977115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:30.319012 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1409/2803] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:30.330970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a18a71-af97-4f17-8a3e-74b131a7203d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:30.333546 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-43a18a71-af97-4f17-8a3e-74b131a7203d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:30.416156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a18a71-af97-4f17-8a3e-74b131a7203d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:30.422244 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354826fe-95a0-40dc-8ab1-5533cafd8122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341358254d884b278d7fa19d95c2dc2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:30.422891 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354826fe-95a0-40dc-8ab1-5533cafd8122 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:30.423166 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354826fe-95a0-40dc-8ab1-5533cafd8122 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:30.423434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354826fe-95a0-40dc-8ab1-5533cafd8122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:30.431101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354826fe-95a0-40dc-8ab1-5533cafd8122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:30.433073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354826fe-95a0-40dc-8ab1-5533cafd8122 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:30.441881 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1395/2804] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:02:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:02:38.807381 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b8c81-5694-432b-bdc6-7fe0bd977115 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:38.869624 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b8c81-5694-432b-bdc6-7fe0bd977115 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:38.875778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:38.877156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:38.877329 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:38.877749 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] PATH_INFO: `/v3/users/a5428cada65b43afa6df79f4bf748c6e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:38.879753 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5428cada65b43afa6df79f4bf748c6e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:38.882193 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:38.932430 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5428cada65b43afa6df79f4bf748c6e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:38.933463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5428cada65b43afa6df79f4bf748c6e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:38.946672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5428cada65b43afa6df79f4bf748c6e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:38.955893 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5428cada65b43afa6df79f4bf748c6e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:38.963578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5428cada65b43afa6df79f4bf748c6e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:38.969639 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1410/2805] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 19:02:38 2022] DELETE /identity/v3/users/a5428cada65b43afa6df79f4bf748c6e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:38.974899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9197c4-7018-4745-9a46-84f535661136 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:38.975436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9197c4-7018-4745-9a46-84f535661136 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:38.975436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9197c4-7018-4745-9a46-84f535661136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:38.996522 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-cc9197c4-7018-4745-9a46-84f535661136 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:38.997778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cc9197c4-7018-4745-9a46-84f535661136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:39.072223 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1396/2806] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:39.085567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:39.087719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:39.164322 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e1dd6-6ce1-4d0b-a35f-a85179c9aec7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:39.169471 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c2cea-3489-45f0-92bb-57bb4999e1e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:39.170122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c2cea-3489-45f0-92bb-57bb4999e1e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:39.170402 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c2cea-3489-45f0-92bb-57bb4999e1e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:39.170676 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c2cea-3489-45f0-92bb-57bb4999e1e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:39.183262 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c2cea-3489-45f0-92bb-57bb4999e1e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:39.184739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c2cea-3489-45f0-92bb-57bb4999e1e9 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:39.198223 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1411/2807] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:02:39.502256 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9197c4-7018-4745-9a46-84f535661136 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:39.504636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cc9197c4-7018-4745-9a46-84f535661136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:39.517135 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9197c4-7018-4745-9a46-84f535661136 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:39.523156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:39.524035 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:39.524316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:39.524619 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] PATH_INFO: `/v3/projects/d0c4f2c188644461881ad70639d5f99c` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:39.533346 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c4f2c188644461881ad70639d5f99c)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:39.535773 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:39.583543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c4f2c188644461881ad70639d5f99c', 'request_id': 'req-ed6b2512-2a6e-43d7-83a5-63930accafd6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:39.591094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c4f2c188644461881ad70639d5f99c', 'request_id': 'req-ed6b2512-2a6e-43d7-83a5-63930accafd6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:39.597830 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c4f2c188644461881ad70639d5f99c', 'request_id': 'req-ed6b2512-2a6e-43d7-83a5-63930accafd6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:39.610106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] The token cache is being invalidate because project d0c4f2c188644461881ad70639d5f99c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 19:02:39.610393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:39.611176 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed6b2512-2a6e-43d7-83a5-63930accafd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:39.613043 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1397/2808] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 19:02:39 2022] DELETE /identity/v3/projects/d0c4f2c188644461881ad70639d5f99c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:39.618436 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ec8ab-e598-4c2f-9bdf-65644c5ca480 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:39.618756 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ec8ab-e598-4c2f-9bdf-65644c5ca480 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:39.619055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ec8ab-e598-4c2f-9bdf-65644c5ca480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:39.637899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c89ec8ab-e598-4c2f-9bdf-65644c5ca480 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:39.639146 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c89ec8ab-e598-4c2f-9bdf-65644c5ca480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:39.700122 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1412/2809] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:39.708096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9243072-20f7-4035-83e3-16603bf4580b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:39.708382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9243072-20f7-4035-83e3-16603bf4580b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:39.708758 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9243072-20f7-4035-83e3-16603bf4580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:39.729080 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b9243072-20f7-4035-83e3-16603bf4580b None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:39.730269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b9243072-20f7-4035-83e3-16603bf4580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:39.739227 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1398/2810] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:39.743540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ec8ab-e598-4c2f-9bdf-65644c5ca480 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:39.753141 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ec8ab-e598-4c2f-9bdf-65644c5ca480 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:39.758636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca90acb-50bf-4fb6-8e3c-c1d3e74afb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:39.759282 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca90acb-50bf-4fb6-8e3c-c1d3e74afb8f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:39.759565 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca90acb-50bf-4fb6-8e3c-c1d3e74afb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:39.759841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca90acb-50bf-4fb6-8e3c-c1d3e74afb8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:39.760589 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca90acb-50bf-4fb6-8e3c-c1d3e74afb8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 19:02:39.760894 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca90acb-50bf-4fb6-8e3c-c1d3e74afb8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:39.762506 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca90acb-50bf-4fb6-8e3c-c1d3e74afb8f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:39.774001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1413/2811] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 19:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 19:02:39.778254 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9243072-20f7-4035-83e3-16603bf4580b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:39.788606 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9243072-20f7-4035-83e3-16603bf4580b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:39.794330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1e4e2-646b-4fe7-b123-f3e1c14106ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:39.794970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1e4e2-646b-4fe7-b123-f3e1c14106ba admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:39.795249 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1e4e2-646b-4fe7-b123-f3e1c14106ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:39.795517 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1e4e2-646b-4fe7-b123-f3e1c14106ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:39.796181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1e4e2-646b-4fe7-b123-f3e1c14106ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:39.798129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1e4e2-646b-4fe7-b123-f3e1c14106ba admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:39.832379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1399/2812] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 19:02:39 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:02:39.836752 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca90acb-50bf-4fb6-8e3c-c1d3e74afb8f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:39.847023 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca90acb-50bf-4fb6-8e3c-c1d3e74afb8f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:39.905490 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ed424-d169-43d6-9222-45e625764153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:39.906155 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ed424-d169-43d6-9222-45e625764153 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:39.906434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ed424-d169-43d6-9222-45e625764153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:39.906701 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ed424-d169-43d6-9222-45e625764153 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:39.907370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ed424-d169-43d6-9222-45e625764153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:39.909308 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ed424-d169-43d6-9222-45e625764153 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:39.929198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1414/2813] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 19:02:39 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:02:39.933809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1e4e2-646b-4fe7-b123-f3e1c14106ba admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:39.945896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1e4e2-646b-4fe7-b123-f3e1c14106ba admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:39.954478 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ec8d1-fb47-43d8-b30a-3f4967410cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:39.955393 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ec8d1-fb47-43d8-b30a-3f4967410cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:39.955784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ec8d1-fb47-43d8-b30a-3f4967410cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:39.956125 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ec8d1-fb47-43d8-b30a-3f4967410cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:39.957691 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ec8d1-fb47-43d8-b30a-3f4967410cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:39.958896 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ec8d1-fb47-43d8-b30a-3f4967410cf1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:39.969409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1400/2814] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:02:39.973693 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ed424-d169-43d6-9222-45e625764153 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:39.984156 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ed424-d169-43d6-9222-45e625764153 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:39.991960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258676c-99f6-4619-8300-dc0f09ac4824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:39.993008 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258676c-99f6-4619-8300-dc0f09ac4824 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:39.993395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258676c-99f6-4619-8300-dc0f09ac4824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:39.993782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258676c-99f6-4619-8300-dc0f09ac4824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:39.994877 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258676c-99f6-4619-8300-dc0f09ac4824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:39.997389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258676c-99f6-4619-8300-dc0f09ac4824 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:40.010050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1415/2815] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Apr 29 19:02:40.014373 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ec8d1-fb47-43d8-b30a-3f4967410cf1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:40.024987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ec8d1-fb47-43d8-b30a-3f4967410cf1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:40.030520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b71ee-49ed-43ad-b676-1b2722ff8bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:40.031386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b71ee-49ed-43ad-b676-1b2722ff8bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:40.031685 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b71ee-49ed-43ad-b676-1b2722ff8bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:40.031950 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b71ee-49ed-43ad-b676-1b2722ff8bbd admin admin] PATH_INFO: `/v3/projects/04f5a541b6764e6db9ec394862eef424/users/5fff796b29294cf5bd868790c72a382e/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:40.046639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b71ee-49ed-43ad-b676-1b2722ff8bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f5a541b6764e6db9ec394862eef424, user_id=5fff796b29294cf5bd868790c72a382e, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:40.049330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b71ee-49ed-43ad-b676-1b2722ff8bbd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:40.066079 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1401/2816] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:02:40 2022] PUT /identity/v3/projects/04f5a541b6764e6db9ec394862eef424/users/5fff796b29294cf5bd868790c72a382e/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:40.076782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c150-7e82-4ffe-b3aa-d9cad170c0ec None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:40.077072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c150-7e82-4ffe-b3aa-d9cad170c0ec None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:40.077347 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c150-7e82-4ffe-b3aa-d9cad170c0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:40.105681 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5357c150-7e82-4ffe-b3aa-d9cad170c0ec None None] MFA Rules not processed for user `5fff796b29294cf5bd868790c72a382e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:40.106880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5357c150-7e82-4ffe-b3aa-d9cad170c0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:40.185252 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1416/2817] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:40.198046 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b71ee-49ed-43ad-b676-1b2722ff8bbd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:40.200099 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-885b71ee-49ed-43ad-b676-1b2722ff8bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:40.256756 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b71ee-49ed-43ad-b676-1b2722ff8bbd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:40.260895 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130588d-36ec-43d1-a828-8ece3ea751bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:40.261529 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130588d-36ec-43d1-a828-8ece3ea751bf service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:40.261804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130588d-36ec-43d1-a828-8ece3ea751bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:40.262102 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130588d-36ec-43d1-a828-8ece3ea751bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:40.270832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130588d-36ec-43d1-a828-8ece3ea751bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:40.272251 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130588d-36ec-43d1-a828-8ece3ea751bf service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:40.280903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1402/2818] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:02:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:02:48.708363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c150-7e82-4ffe-b3aa-d9cad170c0ec None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:48.774545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c150-7e82-4ffe-b3aa-d9cad170c0ec None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:48.780055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:48.780970 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:48.781245 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:48.781513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] PATH_INFO: `/v3/users/5fff796b29294cf5bd868790c72a382e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:48.782727 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fff796b29294cf5bd868790c72a382e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:48.784662 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:48.834058 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fff796b29294cf5bd868790c72a382e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:48.834800 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fff796b29294cf5bd868790c72a382e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:48.848306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fff796b29294cf5bd868790c72a382e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:48.857163 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fff796b29294cf5bd868790c72a382e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:48.864307 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fff796b29294cf5bd868790c72a382e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:48.870278 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1417/2819] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 19:02:48 2022] DELETE /identity/v3/users/5fff796b29294cf5bd868790c72a382e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:48.875267 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68378-b686-4dd0-996b-df7e0ae9de70 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:48.875760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68378-b686-4dd0-996b-df7e0ae9de70 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:48.875760 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68378-b686-4dd0-996b-df7e0ae9de70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:48.893641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7bc68378-b686-4dd0-996b-df7e0ae9de70 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:48.894805 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7bc68378-b686-4dd0-996b-df7e0ae9de70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:48.957663 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1403/2820] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:48.967696 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:48.969217 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:49.027189 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac716d4-d524-435e-9328-ca8bdc3c14a4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:49.031484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b6337-a580-412b-90bb-4669002eb051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:49.032126 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6337-a580-412b-90bb-4669002eb051 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.032379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6337-a580-412b-90bb-4669002eb051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.032684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6337-a580-412b-90bb-4669002eb051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.041979 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b6337-a580-412b-90bb-4669002eb051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:49.043395 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b6337-a580-412b-90bb-4669002eb051 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:49.053085 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1418/2821] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:02:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:02:49.327454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc68378-b686-4dd0-996b-df7e0ae9de70 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:49.329060 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7bc68378-b686-4dd0-996b-df7e0ae9de70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:49.343086 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc68378-b686-4dd0-996b-df7e0ae9de70 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:49.347771 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:49.348699 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.348974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.349245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] PATH_INFO: `/v3/projects/04f5a541b6764e6db9ec394862eef424` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.350434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f5a541b6764e6db9ec394862eef424)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:49.353363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:49.413313 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f5a541b6764e6db9ec394862eef424', 'request_id': 'req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:49.424045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f5a541b6764e6db9ec394862eef424', 'request_id': 'req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:49.430392 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f5a541b6764e6db9ec394862eef424', 'request_id': 'req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:49.441322 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] The token cache is being invalidate because project 04f5a541b6764e6db9ec394862eef424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 19:02:49.441623 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:49.442245 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-0a6ace8d-a392-46f3-ab1c-b199bb226d4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:02:49.443860 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1404/2822] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 19:02:49 2022] DELETE /identity/v3/projects/04f5a541b6764e6db9ec394862eef424 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:49.448953 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d636c68-23d3-4e75-9570-4f1a99946d8a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.449240 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d636c68-23d3-4e75-9570-4f1a99946d8a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.449564 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d636c68-23d3-4e75-9570-4f1a99946d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.468306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9d636c68-23d3-4e75-9570-4f1a99946d8a None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:49.469545 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9d636c68-23d3-4e75-9570-4f1a99946d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:49.530781 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1419/2823] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:49.544678 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbfb3b-8669-4d15-932b-3ac826d23c83 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.545010 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbfb3b-8669-4d15-932b-3ac826d23c83 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.545290 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbfb3b-8669-4d15-932b-3ac826d23c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.565449 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-38fbfb3b-8669-4d15-932b-3ac826d23c83 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:49.566639 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-38fbfb3b-8669-4d15-932b-3ac826d23c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:49.576784 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1405/2824] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:49.581391 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d636c68-23d3-4e75-9570-4f1a99946d8a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:49.592357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d636c68-23d3-4e75-9570-4f1a99946d8a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:49.598038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb851bc-8afa-46aa-ac7f-14bf3eb0d7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:49.598773 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb851bc-8afa-46aa-ac7f-14bf3eb0d7aa admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.599055 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb851bc-8afa-46aa-ac7f-14bf3eb0d7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.599311 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb851bc-8afa-46aa-ac7f-14bf3eb0d7aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.600040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb851bc-8afa-46aa-ac7f-14bf3eb0d7aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 19:02:49.600370 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb851bc-8afa-46aa-ac7f-14bf3eb0d7aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:49.602057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb851bc-8afa-46aa-ac7f-14bf3eb0d7aa admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:49.614431 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1420/2825] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 19:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 19:02:49.619126 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbfb3b-8669-4d15-932b-3ac826d23c83 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:49.628866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbfb3b-8669-4d15-932b-3ac826d23c83 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:49.634751 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac67c7-ebb6-4053-a230-e5b7e0a4f18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:49.635403 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac67c7-ebb6-4053-a230-e5b7e0a4f18c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.635684 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac67c7-ebb6-4053-a230-e5b7e0a4f18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.635923 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac67c7-ebb6-4053-a230-e5b7e0a4f18c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.636661 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac67c7-ebb6-4053-a230-e5b7e0a4f18c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:49.638709 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac67c7-ebb6-4053-a230-e5b7e0a4f18c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:49.667820 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1406/2826] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 19:02:49 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:02:49.672410 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb851bc-8afa-46aa-ac7f-14bf3eb0d7aa admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:49.683614 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb851bc-8afa-46aa-ac7f-14bf3eb0d7aa admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:49.741050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61436dd-158d-4af6-878a-7c930dc449f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:49.741695 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61436dd-158d-4af6-878a-7c930dc449f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.741975 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61436dd-158d-4af6-878a-7c930dc449f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.742279 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61436dd-158d-4af6-878a-7c930dc449f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.742954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61436dd-158d-4af6-878a-7c930dc449f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:49.744878 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61436dd-158d-4af6-878a-7c930dc449f9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:49.770719 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1421/2827] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 19:02:49 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:02:49.776790 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac67c7-ebb6-4053-a230-e5b7e0a4f18c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:49.788443 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac67c7-ebb6-4053-a230-e5b7e0a4f18c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:49.794701 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9e190-ff18-42b3-8842-6418a24e1072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:49.795371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9e190-ff18-42b3-8842-6418a24e1072 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.795650 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9e190-ff18-42b3-8842-6418a24e1072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.795918 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9e190-ff18-42b3-8842-6418a24e1072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.796689 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9e190-ff18-42b3-8842-6418a24e1072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:49.798284 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9e190-ff18-42b3-8842-6418a24e1072 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:49.809429 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1407/2828] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:02:49.814584 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61436dd-158d-4af6-878a-7c930dc449f9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:49.830573 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61436dd-158d-4af6-878a-7c930dc449f9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:49.838650 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1be4b-7852-4da1-a605-89a95aed6e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:49.839591 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1be4b-7852-4da1-a605-89a95aed6e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.839988 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1be4b-7852-4da1-a605-89a95aed6e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.840389 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1be4b-7852-4da1-a605-89a95aed6e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.841441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1be4b-7852-4da1-a605-89a95aed6e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:49.843050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1be4b-7852-4da1-a605-89a95aed6e64 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:49.852990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1422/2829] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:02:49.857835 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9e190-ff18-42b3-8842-6418a24e1072 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:49.868068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9e190-ff18-42b3-8842-6418a24e1072 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:49.874334 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df73ca-4b9c-45dd-9833-649e794b16ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:49.875221 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df73ca-4b9c-45dd-9833-649e794b16ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.875501 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df73ca-4b9c-45dd-9833-649e794b16ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.876082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df73ca-4b9c-45dd-9833-649e794b16ed admin admin] PATH_INFO: `/v3/projects/4698b35230a34d5689ad2b98058a99a7/users/5c169ddb697d4d6bab33076de3aa1804/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.895358 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df73ca-4b9c-45dd-9833-649e794b16ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=4698b35230a34d5689ad2b98058a99a7, user_id=5c169ddb697d4d6bab33076de3aa1804, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:49.898863 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df73ca-4b9c-45dd-9833-649e794b16ed admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:49.915997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1408/2830] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:02:49 2022] PUT /identity/v3/projects/4698b35230a34d5689ad2b98058a99a7/users/5c169ddb697d4d6bab33076de3aa1804/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:49.927754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c09a3a-5d44-4fe5-af0e-2a162c70c9d6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:49.928007 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c09a3a-5d44-4fe5-af0e-2a162c70c9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:49.928413 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c09a3a-5d44-4fe5-af0e-2a162c70c9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:49.958239 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-83c09a3a-5d44-4fe5-af0e-2a162c70c9d6 None None] MFA Rules not processed for user `5c169ddb697d4d6bab33076de3aa1804`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:49.959475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-83c09a3a-5d44-4fe5-af0e-2a162c70c9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:50.029162 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1423/2831] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:49 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 19:02:50.034001 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df73ca-4b9c-45dd-9833-649e794b16ed admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:50.101794 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df73ca-4b9c-45dd-9833-649e794b16ed admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:50.106974 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ea4f4-047b-40b5-b19f-fba6d5cbf6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:50.107627 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ea4f4-047b-40b5-b19f-fba6d5cbf6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:50.107908 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ea4f4-047b-40b5-b19f-fba6d5cbf6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:50.108183 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ea4f4-047b-40b5-b19f-fba6d5cbf6a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:50.108876 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ea4f4-047b-40b5-b19f-fba6d5cbf6a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:50.110809 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ea4f4-047b-40b5-b19f-fba6d5cbf6a8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:50.138235 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1409/2832] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 19:02:50 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:02:50.142173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c09a3a-5d44-4fe5-af0e-2a162c70c9d6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:50.152818 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c09a3a-5d44-4fe5-af0e-2a162c70c9d6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:50.210449 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984125a-3a31-44c9-ae86-7e005cd482ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:50.211137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984125a-3a31-44c9-ae86-7e005cd482ec admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:50.211424 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984125a-3a31-44c9-ae86-7e005cd482ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:50.211700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984125a-3a31-44c9-ae86-7e005cd482ec admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:50.212392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984125a-3a31-44c9-ae86-7e005cd482ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:50.214318 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984125a-3a31-44c9-ae86-7e005cd482ec admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:50.236450 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1424/2833] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 19:02:50 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:02:50.240589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ea4f4-047b-40b5-b19f-fba6d5cbf6a8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:50.251810 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ea4f4-047b-40b5-b19f-fba6d5cbf6a8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:50.256725 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0ba59-a68f-4920-b587-e15c4be8f671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:50.257391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0ba59-a68f-4920-b587-e15c4be8f671 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:50.257781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0ba59-a68f-4920-b587-e15c4be8f671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:50.258057 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0ba59-a68f-4920-b587-e15c4be8f671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:50.258875 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0ba59-a68f-4920-b587-e15c4be8f671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:50.261089 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0ba59-a68f-4920-b587-e15c4be8f671 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:50.274470 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1410/2834] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:02:50.280811 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984125a-3a31-44c9-ae86-7e005cd482ec admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:50.297951 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984125a-3a31-44c9-ae86-7e005cd482ec admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:50.304240 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67074dd0-f250-46f2-b89b-74d3f81bc381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:50.305174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67074dd0-f250-46f2-b89b-74d3f81bc381 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:50.305455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67074dd0-f250-46f2-b89b-74d3f81bc381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:50.305728 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67074dd0-f250-46f2-b89b-74d3f81bc381 admin admin] PATH_INFO: `/v3/projects/bfc7d75720f04aac8d655834d065536b/users/da61943c946d4485a29f087f0a809aa4/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:50.322306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67074dd0-f250-46f2-b89b-74d3f81bc381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc7d75720f04aac8d655834d065536b, user_id=da61943c946d4485a29f087f0a809aa4, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:50.325137 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67074dd0-f250-46f2-b89b-74d3f81bc381 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:50.341593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1425/2835] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:02:50 2022] PUT /identity/v3/projects/bfc7d75720f04aac8d655834d065536b/users/da61943c946d4485a29f087f0a809aa4/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:50.345706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0ba59-a68f-4920-b587-e15c4be8f671 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:50.355811 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0ba59-a68f-4920-b587-e15c4be8f671 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:50.414436 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f6671-7a78-4cac-a5da-2fdda9f60474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:50.415088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f6671-7a78-4cac-a5da-2fdda9f60474 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:50.415365 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f6671-7a78-4cac-a5da-2fdda9f60474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:50.415634 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f6671-7a78-4cac-a5da-2fdda9f60474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:50.416337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f6671-7a78-4cac-a5da-2fdda9f60474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:50.417957 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f6671-7a78-4cac-a5da-2fdda9f60474 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:50.427608 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1411/2836] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:02:50.432142 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67074dd0-f250-46f2-b89b-74d3f81bc381 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:50.443830 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67074dd0-f250-46f2-b89b-74d3f81bc381 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:50.448767 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da96f54-fc06-4427-9f01-6f13349eea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:50.449647 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96f54-fc06-4427-9f01-6f13349eea93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:50.449927 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96f54-fc06-4427-9f01-6f13349eea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:50.450638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da96f54-fc06-4427-9f01-6f13349eea93 admin admin] PATH_INFO: `/v3/projects/bfc7d75720f04aac8d655834d065536b/users/da61943c946d4485a29f087f0a809aa4/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:50.452775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da96f54-fc06-4427-9f01-6f13349eea93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc7d75720f04aac8d655834d065536b, user_id=da61943c946d4485a29f087f0a809aa4, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:50.455392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da96f54-fc06-4427-9f01-6f13349eea93 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:50.465755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1426/2837] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:02:50 2022] PUT /identity/v3/projects/bfc7d75720f04aac8d655834d065536b/users/da61943c946d4485a29f087f0a809aa4/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:02:50.478598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0ea05-1a80-4dd7-85a2-5196a4ce7a6f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:50.478935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0ea05-1a80-4dd7-85a2-5196a4ce7a6f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:50.479316 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0ea05-1a80-4dd7-85a2-5196a4ce7a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:50.513096 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d9a0ea05-1a80-4dd7-85a2-5196a4ce7a6f None None] MFA Rules not processed for user `da61943c946d4485a29f087f0a809aa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:02:50.514575 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d9a0ea05-1a80-4dd7-85a2-5196a4ce7a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:50.590422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1412/2838] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:02:50 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:02:50.602494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da96f54-fc06-4427-9f01-6f13349eea93 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:50.604371 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4da96f54-fc06-4427-9f01-6f13349eea93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:02:50.664805 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da96f54-fc06-4427-9f01-6f13349eea93 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:50.669521 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2e5d0-63ab-41b1-9cb3-df3d75d55d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:50.670192 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e5d0-63ab-41b1-9cb3-df3d75d55d3f service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:50.670466 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e5d0-63ab-41b1-9cb3-df3d75d55d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:50.670739 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e5d0-63ab-41b1-9cb3-df3d75d55d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:50.680475 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2e5d0-63ab-41b1-9cb3-df3d75d55d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:50.681935 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2e5d0-63ab-41b1-9cb3-df3d75d55d3f service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:50.692130 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1427/2839] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:02:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:02:50.779778 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0ea05-1a80-4dd7-85a2-5196a4ce7a6f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:02:50.789703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0ea05-1a80-4dd7-85a2-5196a4ce7a6f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:02:50.793680 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208f0d2-b792-4296-884e-63acf3bd4811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:02:50.794822 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208f0d2-b792-4296-884e-63acf3bd4811 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:02:50.795106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208f0d2-b792-4296-884e-63acf3bd4811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:02:50.795364 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208f0d2-b792-4296-884e-63acf3bd4811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:02:50.851829 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208f0d2-b792-4296-884e-63acf3bd4811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:02:50.853281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208f0d2-b792-4296-884e-63acf3bd4811 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:02:50.862574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1413/2840] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:02:50 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:03:02.267594 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2e5d0-63ab-41b1-9cb3-df3d75d55d3f service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:02.328459 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2e5d0-63ab-41b1-9cb3-df3d75d55d3f service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:02.333687 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:02.334607 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:02.334932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:02.335225 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] PATH_INFO: `/v3/users/5c169ddb697d4d6bab33076de3aa1804` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:02.336416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c169ddb697d4d6bab33076de3aa1804)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:02.338340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:02.391444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c169ddb697d4d6bab33076de3aa1804'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:02.392201 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c169ddb697d4d6bab33076de3aa1804'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:02.409900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c169ddb697d4d6bab33076de3aa1804'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:02.419504 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c169ddb697d4d6bab33076de3aa1804'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:02.426866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c169ddb697d4d6bab33076de3aa1804'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:02.434684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1428/2841] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 19:03:02 2022] DELETE /identity/v3/users/5c169ddb697d4d6bab33076de3aa1804 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:02.439917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af97c1-5373-4709-8ab9-2a1f75c9d1b2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:02.440476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af97c1-5373-4709-8ab9-2a1f75c9d1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:02.440476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af97c1-5373-4709-8ab9-2a1f75c9d1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:02.459948 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-71af97c1-5373-4709-8ab9-2a1f75c9d1b2 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:02.461175 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-71af97c1-5373-4709-8ab9-2a1f75c9d1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:02.521408 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1414/2842] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:02.534293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:02.535952 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:02.595263 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1cf5e-e530-42d2-adae-39e66a55b54d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:02.599247 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e4408-a7d0-4ef5-96ae-c0d87d9e3311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:02.599892 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e4408-a7d0-4ef5-96ae-c0d87d9e3311 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:02.600170 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e4408-a7d0-4ef5-96ae-c0d87d9e3311 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:02.600439 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e4408-a7d0-4ef5-96ae-c0d87d9e3311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:02.611113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e4408-a7d0-4ef5-96ae-c0d87d9e3311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:02.612588 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e4408-a7d0-4ef5-96ae-c0d87d9e3311 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:02.622546 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1429/2843] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:03:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:03:02.913228 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af97c1-5373-4709-8ab9-2a1f75c9d1b2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:02.914833 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-71af97c1-5373-4709-8ab9-2a1f75c9d1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:02.927743 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af97c1-5373-4709-8ab9-2a1f75c9d1b2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:02.933737 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:02.935353 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:02.935800 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:02.936077 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] PATH_INFO: `/v3/projects/4698b35230a34d5689ad2b98058a99a7` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:02.937410 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] RBAC: Authorizing `identity:delete_project(project_id=4698b35230a34d5689ad2b98058a99a7)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:02.939560 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:02.987363 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4698b35230a34d5689ad2b98058a99a7', 'request_id': 'req-5c52eef6-16d4-4ede-87b0-3300167abdda'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:02.995238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4698b35230a34d5689ad2b98058a99a7', 'request_id': 'req-5c52eef6-16d4-4ede-87b0-3300167abdda'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.002270 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4698b35230a34d5689ad2b98058a99a7', 'request_id': 'req-5c52eef6-16d4-4ede-87b0-3300167abdda'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.013118 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] The token cache is being invalidate because project 4698b35230a34d5689ad2b98058a99a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 19:03:03.013391 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.014143 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.015874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1415/2844] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 19:03:02 2022] DELETE /identity/v3/projects/4698b35230a34d5689ad2b98058a99a7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:03.020416 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e4408-a7d0-4ef5-96ae-c0d87d9e3311 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:03.029907 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e4408-a7d0-4ef5-96ae-c0d87d9e3311 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:03.095926 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:03.096839 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:03.097115 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:03.097383 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] PATH_INFO: `/v3/users/da61943c946d4485a29f087f0a809aa4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:03.098583 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da61943c946d4485a29f087f0a809aa4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:03.100479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:03.147385 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da61943c946d4485a29f087f0a809aa4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.148072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da61943c946d4485a29f087f0a809aa4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.160059 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da61943c946d4485a29f087f0a809aa4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.169291 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da61943c946d4485a29f087f0a809aa4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.176276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-72961f08-d1f0-4f28-82a5-e736ff0415e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da61943c946d4485a29f087f0a809aa4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.182947 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1430/2845] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 19:03:03 2022] DELETE /identity/v3/users/da61943c946d4485a29f087f0a809aa4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 19:03:03.462622 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:03.464190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:03.478350 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52eef6-16d4-4ede-87b0-3300167abdda admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:03.535671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:03.536537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:03.536841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:03.537106 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] PATH_INFO: `/v3/projects/bfc7d75720f04aac8d655834d065536b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:03.538302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfc7d75720f04aac8d655834d065536b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:03.540200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:03.586959 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfc7d75720f04aac8d655834d065536b', 'request_id': 'req-d3665980-1cf1-4445-a065-1fd7f2f2bff1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.595573 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfc7d75720f04aac8d655834d065536b', 'request_id': 'req-d3665980-1cf1-4445-a065-1fd7f2f2bff1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.602084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfc7d75720f04aac8d655834d065536b', 'request_id': 'req-d3665980-1cf1-4445-a065-1fd7f2f2bff1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.612469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] The token cache is being invalidate because project bfc7d75720f04aac8d655834d065536b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 19:03:03.612779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.613479 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d3665980-1cf1-4445-a065-1fd7f2f2bff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:03.615084 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1416/2846] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 19:03:03 2022] DELETE /identity/v3/projects/bfc7d75720f04aac8d655834d065536b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:03.620212 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5c4dc-6a9d-4506-8843-43f98d07ca49 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:03.620760 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5c4dc-6a9d-4506-8843-43f98d07ca49 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:03.620862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5c4dc-6a9d-4506-8843-43f98d07ca49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:03.642755 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-57d5c4dc-6a9d-4506-8843-43f98d07ca49 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:03.643909 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-57d5c4dc-6a9d-4506-8843-43f98d07ca49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:03.715340 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1431/2847] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:03.723702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64105973-248a-44dc-97cb-5845c565bd27 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:03.723987 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64105973-248a-44dc-97cb-5845c565bd27 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:03.724258 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64105973-248a-44dc-97cb-5845c565bd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:03.744259 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-64105973-248a-44dc-97cb-5845c565bd27 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:03.745474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-64105973-248a-44dc-97cb-5845c565bd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:03.754239 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1417/2848] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:03.758700 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5c4dc-6a9d-4506-8843-43f98d07ca49 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:03.769561 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5c4dc-6a9d-4506-8843-43f98d07ca49 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:03.774735 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5506167-e3a9-4c9d-80e1-3bfbab81e568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:03.775399 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5506167-e3a9-4c9d-80e1-3bfbab81e568 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:03.775679 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5506167-e3a9-4c9d-80e1-3bfbab81e568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:03.775956 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5506167-e3a9-4c9d-80e1-3bfbab81e568 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:03.776725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5506167-e3a9-4c9d-80e1-3bfbab81e568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 19:03:03.777038 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5506167-e3a9-4c9d-80e1-3bfbab81e568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:03.778629 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5506167-e3a9-4c9d-80e1-3bfbab81e568 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:03.791113 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1432/2849] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 19:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 19:03:03.795587 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64105973-248a-44dc-97cb-5845c565bd27 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:03.805180 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64105973-248a-44dc-97cb-5845c565bd27 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:03.811454 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70528d-3156-4ead-9b60-1258d98b3675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:03.812103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70528d-3156-4ead-9b60-1258d98b3675 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:03.812388 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70528d-3156-4ead-9b60-1258d98b3675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:03.813018 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70528d-3156-4ead-9b60-1258d98b3675 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:03.813702 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70528d-3156-4ead-9b60-1258d98b3675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:03.815585 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70528d-3156-4ead-9b60-1258d98b3675 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:03.843694 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1418/2850] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 19:03:03 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:03:03.848337 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5506167-e3a9-4c9d-80e1-3bfbab81e568 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:03.858482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5506167-e3a9-4c9d-80e1-3bfbab81e568 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:03.921682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e04f56-eaeb-43bb-8d36-c6598e27c579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:03.922312 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e04f56-eaeb-43bb-8d36-c6598e27c579 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:03.922628 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e04f56-eaeb-43bb-8d36-c6598e27c579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:03.922901 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e04f56-eaeb-43bb-8d36-c6598e27c579 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:03.923587 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e04f56-eaeb-43bb-8d36-c6598e27c579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:03.925540 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e04f56-eaeb-43bb-8d36-c6598e27c579 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:03.951656 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1433/2851] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 19:03:03 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 19:03:03.955874 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70528d-3156-4ead-9b60-1258d98b3675 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:03.966662 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70528d-3156-4ead-9b60-1258d98b3675 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:03.971713 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf1a3e-e2af-4585-8326-b529948b8acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:03.972371 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf1a3e-e2af-4585-8326-b529948b8acd admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:03.972671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf1a3e-e2af-4585-8326-b529948b8acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:03.972938 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf1a3e-e2af-4585-8326-b529948b8acd admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:03.973671 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf1a3e-e2af-4585-8326-b529948b8acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:03.975242 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf1a3e-e2af-4585-8326-b529948b8acd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:03.985348 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1419/2852] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:03:03.990718 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e04f56-eaeb-43bb-8d36-c6598e27c579 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:04.000404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e04f56-eaeb-43bb-8d36-c6598e27c579 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:04.005312 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7fc07-2db8-453a-a17d-7780b3e5ac01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:04.005963 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7fc07-2db8-453a-a17d-7780b3e5ac01 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.006238 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7fc07-2db8-453a-a17d-7780b3e5ac01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.006502 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7fc07-2db8-453a-a17d-7780b3e5ac01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.007215 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7fc07-2db8-453a-a17d-7780b3e5ac01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:04.008868 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7fc07-2db8-453a-a17d-7780b3e5ac01 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:04.018782 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1434/2853] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:03:04.022917 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf1a3e-e2af-4585-8326-b529948b8acd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:04.033071 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf1a3e-e2af-4585-8326-b529948b8acd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:04.039876 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed5276-829d-4775-b4b3-3948fa45ab75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:04.041302 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed5276-829d-4775-b4b3-3948fa45ab75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.041682 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed5276-829d-4775-b4b3-3948fa45ab75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.042045 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed5276-829d-4775-b4b3-3948fa45ab75 admin admin] PATH_INFO: `/v3/projects/81e5d1b20d324a9685a50e3bf510efa2/users/3ef570f4baa4424e99992bedfee16938/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.058520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed5276-829d-4775-b4b3-3948fa45ab75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e5d1b20d324a9685a50e3bf510efa2, user_id=3ef570f4baa4424e99992bedfee16938, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:04.061299 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed5276-829d-4775-b4b3-3948fa45ab75 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:04.074533 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1420/2854] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:03:04 2022] PUT /identity/v3/projects/81e5d1b20d324a9685a50e3bf510efa2/users/3ef570f4baa4424e99992bedfee16938/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 19:03:04.084964 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6feda2-16c6-4547-8c45-302607aac5b9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.085261 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6feda2-16c6-4547-8c45-302607aac5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.085541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6feda2-16c6-4547-8c45-302607aac5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.112509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7a6feda2-16c6-4547-8c45-302607aac5b9 None None] MFA Rules not processed for user `3ef570f4baa4424e99992bedfee16938`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:04.113734 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7a6feda2-16c6-4547-8c45-302607aac5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:04.191966 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1435/2855] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:04.197280 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed5276-829d-4775-b4b3-3948fa45ab75 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:04.270261 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed5276-829d-4775-b4b3-3948fa45ab75 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:04.275504 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3272fb-c73b-4b8e-83af-72162ab91e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:04.276154 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3272fb-c73b-4b8e-83af-72162ab91e65 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.276434 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3272fb-c73b-4b8e-83af-72162ab91e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.276746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3272fb-c73b-4b8e-83af-72162ab91e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.277428 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3272fb-c73b-4b8e-83af-72162ab91e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:04.279326 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3272fb-c73b-4b8e-83af-72162ab91e65 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:04.307052 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1421/2856] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 19:03:04 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:03:04.313750 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6feda2-16c6-4547-8c45-302607aac5b9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:04.325045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6feda2-16c6-4547-8c45-302607aac5b9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:04.394005 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a61ff-9e51-4144-9e00-83fa9d3d5105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:04.394682 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a61ff-9e51-4144-9e00-83fa9d3d5105 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.394961 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a61ff-9e51-4144-9e00-83fa9d3d5105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.395232 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a61ff-9e51-4144-9e00-83fa9d3d5105 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.395929 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a61ff-9e51-4144-9e00-83fa9d3d5105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:04.397880 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a61ff-9e51-4144-9e00-83fa9d3d5105 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:04.420903 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1436/2857] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 19:03:04 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 19:03:04.425204 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3272fb-c73b-4b8e-83af-72162ab91e65 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:04.436069 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3272fb-c73b-4b8e-83af-72162ab91e65 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:04.440832 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4dec0-3f6e-4c40-becd-c89e511c9bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:04.441492 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4dec0-3f6e-4c40-becd-c89e511c9bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.441769 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4dec0-3f6e-4c40-becd-c89e511c9bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.442091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4dec0-3f6e-4c40-becd-c89e511c9bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.442761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4dec0-3f6e-4c40-becd-c89e511c9bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:04.444349 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4dec0-3f6e-4c40-becd-c89e511c9bc2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:04.454831 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1422/2858] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:03:04.459172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a61ff-9e51-4144-9e00-83fa9d3d5105 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:04.469576 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a61ff-9e51-4144-9e00-83fa9d3d5105 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:04.474509 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312043c-ff0c-4068-95a2-445563aa7f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:04.475387 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312043c-ff0c-4068-95a2-445563aa7f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.475664 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312043c-ff0c-4068-95a2-445563aa7f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.475932 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312043c-ff0c-4068-95a2-445563aa7f95 admin admin] PATH_INFO: `/v3/projects/e67a12314cdf4c8fa2d803ab5247aefd/users/5489f847b759496aaf40dd4bce3ad59f/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.492631 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312043c-ff0c-4068-95a2-445563aa7f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67a12314cdf4c8fa2d803ab5247aefd, user_id=5489f847b759496aaf40dd4bce3ad59f, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:04.495294 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312043c-ff0c-4068-95a2-445563aa7f95 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:04.509666 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1437/2859] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:03:04 2022] PUT /identity/v3/projects/e67a12314cdf4c8fa2d803ab5247aefd/users/5489f847b759496aaf40dd4bce3ad59f/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:04.514814 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4dec0-3f6e-4c40-becd-c89e511c9bc2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:04.525626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4dec0-3f6e-4c40-becd-c89e511c9bc2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:04.591332 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c7349-ca73-4422-bdc6-927aab15a758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:04.591997 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c7349-ca73-4422-bdc6-927aab15a758 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.592275 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c7349-ca73-4422-bdc6-927aab15a758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.592572 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c7349-ca73-4422-bdc6-927aab15a758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.593287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c7349-ca73-4422-bdc6-927aab15a758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:04.595224 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c7349-ca73-4422-bdc6-927aab15a758 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:04.605169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1423/2860] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:03:04.609585 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312043c-ff0c-4068-95a2-445563aa7f95 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:04.619981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312043c-ff0c-4068-95a2-445563aa7f95 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:04.625025 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2865a-be9f-4f0e-9aba-79b8cd710a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:04.625857 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2865a-be9f-4f0e-9aba-79b8cd710a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.626132 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2865a-be9f-4f0e-9aba-79b8cd710a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.626392 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2865a-be9f-4f0e-9aba-79b8cd710a5a admin admin] PATH_INFO: `/v3/projects/e67a12314cdf4c8fa2d803ab5247aefd/users/5489f847b759496aaf40dd4bce3ad59f/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.628477 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2865a-be9f-4f0e-9aba-79b8cd710a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67a12314cdf4c8fa2d803ab5247aefd, user_id=5489f847b759496aaf40dd4bce3ad59f, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:04.631131 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2865a-be9f-4f0e-9aba-79b8cd710a5a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:04.641862 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1438/2861] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:03:04 2022] PUT /identity/v3/projects/e67a12314cdf4c8fa2d803ab5247aefd/users/5489f847b759496aaf40dd4bce3ad59f/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:04.703481 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a31198-e999-4386-8bf4-c250c13606af None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.703747 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a31198-e999-4386-8bf4-c250c13606af None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.704025 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a31198-e999-4386-8bf4-c250c13606af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.734289 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f2a31198-e999-4386-8bf4-c250c13606af None None] MFA Rules not processed for user `5489f847b759496aaf40dd4bce3ad59f`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:04.736030 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f2a31198-e999-4386-8bf4-c250c13606af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:04.809215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1424/2862] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:04 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:04.820992 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2865a-be9f-4f0e-9aba-79b8cd710a5a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:04.822954 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4ea2865a-be9f-4f0e-9aba-79b8cd710a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:04.881902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2865a-be9f-4f0e-9aba-79b8cd710a5a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:04.886703 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727909b-a239-4845-bf38-58ca33562217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:04.887355 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727909b-a239-4845-bf38-58ca33562217 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.887638 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727909b-a239-4845-bf38-58ca33562217 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.887915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727909b-a239-4845-bf38-58ca33562217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:04.899338 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727909b-a239-4845-bf38-58ca33562217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:04.900812 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727909b-a239-4845-bf38-58ca33562217 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:04.911234 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1439/2863] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:03:04 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:03:04.961636 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a31198-e999-4386-8bf4-c250c13606af None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:04.963112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f2a31198-e999-4386-8bf4-c250c13606af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:04.976128 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a31198-e999-4386-8bf4-c250c13606af None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:04.980781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572af11-0a1d-4963-8bdd-75c7d5481aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:04.981422 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572af11-0a1d-4963-8bdd-75c7d5481aae service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:04.981703 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572af11-0a1d-4963-8bdd-75c7d5481aae service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:04.981972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572af11-0a1d-4963-8bdd-75c7d5481aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:05.038441 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572af11-0a1d-4963-8bdd-75c7d5481aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:05.039984 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572af11-0a1d-4963-8bdd-75c7d5481aae service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:05.048914 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1425/2864] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:03:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:03:08.416685 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727909b-a239-4845-bf38-58ca33562217 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:08.480403 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727909b-a239-4845-bf38-58ca33562217 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:08.487106 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:08.487985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:08.488290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:08.488593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] PATH_INFO: `/v3/users/3ef570f4baa4424e99992bedfee16938` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:08.489915 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ef570f4baa4424e99992bedfee16938)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:08.491842 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:08.551972 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ef570f4baa4424e99992bedfee16938'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:08.553006 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ef570f4baa4424e99992bedfee16938'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:08.569646 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ef570f4baa4424e99992bedfee16938'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:08.578902 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ef570f4baa4424e99992bedfee16938'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:08.588026 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ef570f4baa4424e99992bedfee16938'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:08.596173 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1440/2865] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 19:03:08 2022] DELETE /identity/v3/users/3ef570f4baa4424e99992bedfee16938 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:08.600924 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c75f32-1763-4f29-bcd7-82975780b8d6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:08.601469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c75f32-1763-4f29-bcd7-82975780b8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:08.601469 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c75f32-1763-4f29-bcd7-82975780b8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:08.625547 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-65c75f32-1763-4f29-bcd7-82975780b8d6 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:08.626728 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-65c75f32-1763-4f29-bcd7-82975780b8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:08.693397 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1426/2866] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:08.705281 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:08.706813 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:08.766567 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c703912-f1e9-409a-a272-d4f3eee6a17c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:08.771686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f950f-2230-4965-b8dc-0713b110751a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:08.772352 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f950f-2230-4965-b8dc-0713b110751a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:08.772669 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f950f-2230-4965-b8dc-0713b110751a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:08.772942 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f950f-2230-4965-b8dc-0713b110751a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:08.783168 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f950f-2230-4965-b8dc-0713b110751a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:08.784651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f950f-2230-4965-b8dc-0713b110751a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:08.794198 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1441/2867] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:03:09.071551 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c75f32-1763-4f29-bcd7-82975780b8d6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:09.073051 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-65c75f32-1763-4f29-bcd7-82975780b8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:09.087386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c75f32-1763-4f29-bcd7-82975780b8d6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:09.093543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:09.095132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:09.095414 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:09.095789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] PATH_INFO: `/v3/projects/81e5d1b20d324a9685a50e3bf510efa2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:09.097013 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e5d1b20d324a9685a50e3bf510efa2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:09.099101 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:09.147626 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e5d1b20d324a9685a50e3bf510efa2', 'request_id': 'req-1de3724a-c643-4364-a081-f6f875c402a6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.157082 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e5d1b20d324a9685a50e3bf510efa2', 'request_id': 'req-1de3724a-c643-4364-a081-f6f875c402a6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.163475 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e5d1b20d324a9685a50e3bf510efa2', 'request_id': 'req-1de3724a-c643-4364-a081-f6f875c402a6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.174491 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] The token cache is being invalidate because project 81e5d1b20d324a9685a50e3bf510efa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 19:03:09.174749 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.175413 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.177032 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1427/2868] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 19:03:09 2022] DELETE /identity/v3/projects/81e5d1b20d324a9685a50e3bf510efa2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:09.181494 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f950f-2230-4965-b8dc-0713b110751a service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:09.192159 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f950f-2230-4965-b8dc-0713b110751a service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:09.248577 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:09.249453 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:09.249769 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:09.250045 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] PATH_INFO: `/v3/users/5489f847b759496aaf40dd4bce3ad59f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:09.251480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5489f847b759496aaf40dd4bce3ad59f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:09.253480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:09.300297 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5489f847b759496aaf40dd4bce3ad59f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.301074 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5489f847b759496aaf40dd4bce3ad59f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.314437 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5489f847b759496aaf40dd4bce3ad59f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.323230 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5489f847b759496aaf40dd4bce3ad59f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.330775 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4391c1df-87f6-456e-ba11-26e259588020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5489f847b759496aaf40dd4bce3ad59f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.336725 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1442/2869] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 19:03:09 2022] DELETE /identity/v3/users/5489f847b759496aaf40dd4bce3ad59f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:09.605132 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:09.606780 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:09.619861 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3724a-c643-4364-a081-f6f875c402a6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:09.678292 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:09.679153 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:09.679476 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:09.679746 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] PATH_INFO: `/v3/projects/e67a12314cdf4c8fa2d803ab5247aefd` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:09.680960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e67a12314cdf4c8fa2d803ab5247aefd)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:09.683229 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:09.730723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e67a12314cdf4c8fa2d803ab5247aefd', 'request_id': 'req-09b4c204-5b75-4148-ad93-ac3dead18b6e'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.741134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e67a12314cdf4c8fa2d803ab5247aefd', 'request_id': 'req-09b4c204-5b75-4148-ad93-ac3dead18b6e'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.747416 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e67a12314cdf4c8fa2d803ab5247aefd', 'request_id': 'req-09b4c204-5b75-4148-ad93-ac3dead18b6e'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.758883 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] The token cache is being invalidate because project e67a12314cdf4c8fa2d803ab5247aefd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 19:03:09.759199 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.760207 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-09b4c204-5b75-4148-ad93-ac3dead18b6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:09.762656 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1428/2870] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 19:03:09 2022] DELETE /identity/v3/projects/e67a12314cdf4c8fa2d803ab5247aefd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:09.767699 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996bfcc-4af6-49ec-9e2c-dce4bb553744 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:09.768211 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996bfcc-4af6-49ec-9e2c-dce4bb553744 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:09.768643 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996bfcc-4af6-49ec-9e2c-dce4bb553744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:09.788526 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6996bfcc-4af6-49ec-9e2c-dce4bb553744 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:09.789777 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6996bfcc-4af6-49ec-9e2c-dce4bb553744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:09.854094 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1443/2871] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:09.865269 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640108a-4cb9-4d41-ab06-5f2228549ffa None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:09.865600 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640108a-4cb9-4d41-ab06-5f2228549ffa None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:09.865882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640108a-4cb9-4d41-ab06-5f2228549ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:09.884751 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f640108a-4cb9-4d41-ab06-5f2228549ffa None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:09.885960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f640108a-4cb9-4d41-ab06-5f2228549ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:09.894616 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1429/2872] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Apr 29 19:03:09.898990 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996bfcc-4af6-49ec-9e2c-dce4bb553744 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:09.909265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996bfcc-4af6-49ec-9e2c-dce4bb553744 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:09.914853 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ace50d-054d-4a7f-942d-bcaa02776120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:09.915513 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ace50d-054d-4a7f-942d-bcaa02776120 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:09.915794 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ace50d-054d-4a7f-942d-bcaa02776120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:09.916089 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ace50d-054d-4a7f-942d-bcaa02776120 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:09.916872 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ace50d-054d-4a7f-942d-bcaa02776120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 19:03:09.917182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ace50d-054d-4a7f-942d-bcaa02776120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:09.918776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ace50d-054d-4a7f-942d-bcaa02776120 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:09.932888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1444/2873] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 19:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 19:03:09.937409 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640108a-4cb9-4d41-ab06-5f2228549ffa None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:09.948281 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640108a-4cb9-4d41-ab06-5f2228549ffa None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:09.957653 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a690b-50f0-4443-b085-6c30ebc280d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:09.958317 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a690b-50f0-4443-b085-6c30ebc280d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:09.958588 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a690b-50f0-4443-b085-6c30ebc280d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:09.958857 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a690b-50f0-4443-b085-6c30ebc280d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:09.959538 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a690b-50f0-4443-b085-6c30ebc280d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:09.961524 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a690b-50f0-4443-b085-6c30ebc280d9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:09.991995 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1430/2874] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 19:03:09 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:03:09.996559 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ace50d-054d-4a7f-942d-bcaa02776120 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.008826 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ace50d-054d-4a7f-942d-bcaa02776120 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:10.069455 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617550d9-20db-4b2c-bb58-8e696bcdf9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:10.070124 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617550d9-20db-4b2c-bb58-8e696bcdf9eb admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.070408 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617550d9-20db-4b2c-bb58-8e696bcdf9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.070679 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617550d9-20db-4b2c-bb58-8e696bcdf9eb admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.071381 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617550d9-20db-4b2c-bb58-8e696bcdf9eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:10.073333 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617550d9-20db-4b2c-bb58-8e696bcdf9eb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:10.094218 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1445/2875] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 19:03:09 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 19:03:10.099535 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a690b-50f0-4443-b085-6c30ebc280d9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.110342 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a690b-50f0-4443-b085-6c30ebc280d9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:10.115287 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9cf76-eb79-45d4-82eb-d8c95572b177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:10.115935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9cf76-eb79-45d4-82eb-d8c95572b177 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.116215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9cf76-eb79-45d4-82eb-d8c95572b177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.116627 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9cf76-eb79-45d4-82eb-d8c95572b177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.117347 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9cf76-eb79-45d4-82eb-d8c95572b177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:10.118935 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9cf76-eb79-45d4-82eb-d8c95572b177 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:10.129372 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1431/2876] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:03:10.135256 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617550d9-20db-4b2c-bb58-8e696bcdf9eb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.147684 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617550d9-20db-4b2c-bb58-8e696bcdf9eb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:10.153114 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fecd74-d518-49ea-942b-d840808364a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:10.153776 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fecd74-d518-49ea-942b-d840808364a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.154063 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fecd74-d518-49ea-942b-d840808364a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.154339 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fecd74-d518-49ea-942b-d840808364a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.155065 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fecd74-d518-49ea-942b-d840808364a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:10.156686 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fecd74-d518-49ea-942b-d840808364a6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:10.167195 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1446/2877] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:03:10.171525 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9cf76-eb79-45d4-82eb-d8c95572b177 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.182798 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9cf76-eb79-45d4-82eb-d8c95572b177 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:10.187960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee106470-c5bd-4a39-b36a-37e080e60c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:10.188882 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee106470-c5bd-4a39-b36a-37e080e60c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.189169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee106470-c5bd-4a39-b36a-37e080e60c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.189444 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee106470-c5bd-4a39-b36a-37e080e60c7b admin admin] PATH_INFO: `/v3/projects/5986f26117c6418ca6e9e92464bfbe99/users/1067b7059a574b4cb8a2c7f926567f51/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.203941 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee106470-c5bd-4a39-b36a-37e080e60c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5986f26117c6418ca6e9e92464bfbe99, user_id=1067b7059a574b4cb8a2c7f926567f51, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:10.206624 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee106470-c5bd-4a39-b36a-37e080e60c7b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:10.220515 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1432/2878] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:03:10 2022] PUT /identity/v3/projects/5986f26117c6418ca6e9e92464bfbe99/users/1067b7059a574b4cb8a2c7f926567f51/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:10.230866 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf8dbc-979f-4e5a-adfd-73791189c620 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.231167 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf8dbc-979f-4e5a-adfd-73791189c620 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.231444 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf8dbc-979f-4e5a-adfd-73791189c620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.258050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0bdf8dbc-979f-4e5a-adfd-73791189c620 None None] MFA Rules not processed for user `1067b7059a574b4cb8a2c7f926567f51`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:10.259246 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0bdf8dbc-979f-4e5a-adfd-73791189c620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:10.349331 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1447/2879] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:10.354088 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee106470-c5bd-4a39-b36a-37e080e60c7b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.424589 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee106470-c5bd-4a39-b36a-37e080e60c7b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:10.430238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922c42c-0afa-4efd-a7ef-631dd7420e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:10.430903 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922c42c-0afa-4efd-a7ef-631dd7420e10 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.431187 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922c42c-0afa-4efd-a7ef-631dd7420e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.431457 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922c42c-0afa-4efd-a7ef-631dd7420e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.432539 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922c42c-0afa-4efd-a7ef-631dd7420e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:10.434458 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922c42c-0afa-4efd-a7ef-631dd7420e10 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:10.466669 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1433/2880] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 19:03:10 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:03:10.471580 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf8dbc-979f-4e5a-adfd-73791189c620 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.488384 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf8dbc-979f-4e5a-adfd-73791189c620 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:10.558690 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad80d78-6790-4570-83fd-3122da2948b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:10.559363 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad80d78-6790-4570-83fd-3122da2948b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.559647 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad80d78-6790-4570-83fd-3122da2948b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.559922 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad80d78-6790-4570-83fd-3122da2948b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.560633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad80d78-6790-4570-83fd-3122da2948b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:10.562599 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad80d78-6790-4570-83fd-3122da2948b7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:10.586674 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1448/2881] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 19:03:10 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:03:10.591693 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922c42c-0afa-4efd-a7ef-631dd7420e10 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.603140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922c42c-0afa-4efd-a7ef-631dd7420e10 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:10.608140 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ab38a-75a2-41e8-b2a5-d129e9b48e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:10.608838 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ab38a-75a2-41e8-b2a5-d129e9b48e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.609116 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ab38a-75a2-41e8-b2a5-d129e9b48e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.609385 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ab38a-75a2-41e8-b2a5-d129e9b48e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.610534 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ab38a-75a2-41e8-b2a5-d129e9b48e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:10.611866 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ab38a-75a2-41e8-b2a5-d129e9b48e89 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:10.622264 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1434/2882] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:03:10.627176 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad80d78-6790-4570-83fd-3122da2948b7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.638086 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad80d78-6790-4570-83fd-3122da2948b7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:10.644107 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6587f70-eba0-4729-9a46-4457a5733973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:10.645024 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6587f70-eba0-4729-9a46-4457a5733973 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.645302 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6587f70-eba0-4729-9a46-4457a5733973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.645574 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6587f70-eba0-4729-9a46-4457a5733973 admin admin] PATH_INFO: `/v3/projects/8452d964af4242deb300d852e13e429b/users/dabe3cb9b5a447818be4aa22627dec61/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.661605 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6587f70-eba0-4729-9a46-4457a5733973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8452d964af4242deb300d852e13e429b, user_id=dabe3cb9b5a447818be4aa22627dec61, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:10.664221 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6587f70-eba0-4729-9a46-4457a5733973 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:10.679379 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1449/2883] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:03:10 2022] PUT /identity/v3/projects/8452d964af4242deb300d852e13e429b/users/dabe3cb9b5a447818be4aa22627dec61/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:10.683761 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ab38a-75a2-41e8-b2a5-d129e9b48e89 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.694257 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ab38a-75a2-41e8-b2a5-d129e9b48e89 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:10.754706 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818247f-f091-492f-b271-1157eb3596c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:10.755377 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818247f-f091-492f-b271-1157eb3596c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.755658 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818247f-f091-492f-b271-1157eb3596c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.755869 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818247f-f091-492f-b271-1157eb3596c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.756632 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818247f-f091-492f-b271-1157eb3596c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:10.758224 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818247f-f091-492f-b271-1157eb3596c7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:10.767888 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1435/2884] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:03:10.772182 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6587f70-eba0-4729-9a46-4457a5733973 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.782897 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6587f70-eba0-4729-9a46-4457a5733973 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:10.788441 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b1be1-3942-4865-81c2-5910498fbcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:10.789357 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b1be1-3942-4865-81c2-5910498fbcd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.789651 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b1be1-3942-4865-81c2-5910498fbcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.789918 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b1be1-3942-4865-81c2-5910498fbcd8 admin admin] PATH_INFO: `/v3/projects/8452d964af4242deb300d852e13e429b/users/dabe3cb9b5a447818be4aa22627dec61/roles/e55ef11844a742f5889243f189bd59bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.791959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b1be1-3942-4865-81c2-5910498fbcd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8452d964af4242deb300d852e13e429b, user_id=dabe3cb9b5a447818be4aa22627dec61, role_id=e55ef11844a742f5889243f189bd59bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:10.794633 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b1be1-3942-4865-81c2-5910498fbcd8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:10.805260 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1450/2885] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:03:10 2022] PUT /identity/v3/projects/8452d964af4242deb300d852e13e429b/users/dabe3cb9b5a447818be4aa22627dec61/roles/e55ef11844a742f5889243f189bd59bc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:10.817390 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340041-1b92-43e1-be92-8ae8a8a5f065 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:10.817698 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340041-1b92-43e1-be92-8ae8a8a5f065 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:10.817972 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340041-1b92-43e1-be92-8ae8a8a5f065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:10.845480 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0a340041-1b92-43e1-be92-8ae8a8a5f065 None None] MFA Rules not processed for user `dabe3cb9b5a447818be4aa22627dec61`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:10.846753 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0a340041-1b92-43e1-be92-8ae8a8a5f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:10.914473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1436/2886] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:10.926431 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b1be1-3942-4865-81c2-5910498fbcd8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:10.929174 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7b5b1be1-3942-4865-81c2-5910498fbcd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:10.995094 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b1be1-3942-4865-81c2-5910498fbcd8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:11.000075 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d026cc2-4797-4758-9dc7-69b6526ef85a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:11.000899 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d026cc2-4797-4758-9dc7-69b6526ef85a service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:11.001226 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d026cc2-4797-4758-9dc7-69b6526ef85a service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:11.001484 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d026cc2-4797-4758-9dc7-69b6526ef85a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:11.011320 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d026cc2-4797-4758-9dc7-69b6526ef85a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:11.012788 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d026cc2-4797-4758-9dc7-69b6526ef85a service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:11.023057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1451/2887] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:03:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:03:11.081382 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340041-1b92-43e1-be92-8ae8a8a5f065 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:11.083719 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0a340041-1b92-43e1-be92-8ae8a8a5f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:11.103499 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340041-1b92-43e1-be92-8ae8a8a5f065 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:11.109848 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca9967-21cd-484e-9a93-e4bec282a683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:11.111094 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca9967-21cd-484e-9a93-e4bec282a683 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:11.111376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca9967-21cd-484e-9a93-e4bec282a683 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:11.111641 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca9967-21cd-484e-9a93-e4bec282a683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:11.168038 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca9967-21cd-484e-9a93-e4bec282a683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:11.169502 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca9967-21cd-484e-9a93-e4bec282a683 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:11.179091 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1437/2888] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:03:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:03:14.510710 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d026cc2-4797-4758-9dc7-69b6526ef85a service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:14.578593 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d026cc2-4797-4758-9dc7-69b6526ef85a service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:14.583635 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:14.584530 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:14.584819 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:14.585090 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] PATH_INFO: `/v3/users/1067b7059a574b4cb8a2c7f926567f51` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:14.586287 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1067b7059a574b4cb8a2c7f926567f51)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:14.588188 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:14.638404 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1067b7059a574b4cb8a2c7f926567f51'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:14.639184 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1067b7059a574b4cb8a2c7f926567f51'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:14.652672 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1067b7059a574b4cb8a2c7f926567f51'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:14.661140 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1067b7059a574b4cb8a2c7f926567f51'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:14.668001 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1067b7059a574b4cb8a2c7f926567f51'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:14.673747 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1452/2889] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 19:03:14 2022] DELETE /identity/v3/users/1067b7059a574b4cb8a2c7f926567f51 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:14.678330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78c99b-cff2-4586-b22e-9a5430a51afc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:14.678842 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78c99b-cff2-4586-b22e-9a5430a51afc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:14.678961 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78c99b-cff2-4586-b22e-9a5430a51afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:14.697657 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-2a78c99b-cff2-4586-b22e-9a5430a51afc None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:14.698828 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2a78c99b-cff2-4586-b22e-9a5430a51afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:14.763081 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1438/2890] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:14.774290 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:14.775741 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:14.837010 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84882b4b-b379-4a16-9803-180d5bd4bcfd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:14.841043 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a980f-c3d5-480d-b0d4-e840617f7935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:14.841688 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a980f-c3d5-480d-b0d4-e840617f7935 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:14.841971 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a980f-c3d5-480d-b0d4-e840617f7935 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:14.842240 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a980f-c3d5-480d-b0d4-e840617f7935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:14.851713 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a980f-c3d5-480d-b0d4-e840617f7935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:14.853181 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a980f-c3d5-480d-b0d4-e840617f7935 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:14.862660 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1453/2891] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:03:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:03:15.127170 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78c99b-cff2-4586-b22e-9a5430a51afc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:15.128621 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2a78c99b-cff2-4586-b22e-9a5430a51afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:15.145181 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78c99b-cff2-4586-b22e-9a5430a51afc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:15.151594 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:15.152849 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:15.153188 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:15.153723 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] PATH_INFO: `/v3/projects/5986f26117c6418ca6e9e92464bfbe99` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:15.155156 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5986f26117c6418ca6e9e92464bfbe99)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:15.157133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:15.204668 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5986f26117c6418ca6e9e92464bfbe99', 'request_id': 'req-f9eb4e14-17f2-467e-834d-40213444b89c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:15.213456 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5986f26117c6418ca6e9e92464bfbe99', 'request_id': 'req-f9eb4e14-17f2-467e-834d-40213444b89c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:15.220339 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5986f26117c6418ca6e9e92464bfbe99', 'request_id': 'req-f9eb4e14-17f2-467e-834d-40213444b89c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:15.233129 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] The token cache is being invalidate because project 5986f26117c6418ca6e9e92464bfbe99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 19:03:15.233537 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:15.234631 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:15.237100 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1439/2892] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 19:03:15 2022] DELETE /identity/v3/projects/5986f26117c6418ca6e9e92464bfbe99 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:15.241349 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a980f-c3d5-480d-b0d4-e840617f7935 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:15.251911 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a980f-c3d5-480d-b0d4-e840617f7935 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:15.308458 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:15.309362 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:15.309642 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:15.309910 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] PATH_INFO: `/v3/users/dabe3cb9b5a447818be4aa22627dec61` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:15.311933 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dabe3cb9b5a447818be4aa22627dec61)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:15.315299 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:15.367923 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dabe3cb9b5a447818be4aa22627dec61'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:15.368636 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dabe3cb9b5a447818be4aa22627dec61'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:15.380888 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dabe3cb9b5a447818be4aa22627dec61'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:15.389959 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dabe3cb9b5a447818be4aa22627dec61'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:15.397849 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-faea2250-b166-492a-8290-6e84391751b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dabe3cb9b5a447818be4aa22627dec61'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:15.404136 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1454/2893] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 19:03:15 2022] DELETE /identity/v3/users/dabe3cb9b5a447818be4aa22627dec61 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:15.960994 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:15.962607 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:15.976200 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb4e14-17f2-467e-834d-40213444b89c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:16.031655 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:16.032427 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.032789 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.033520 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] PATH_INFO: `/v3/projects/8452d964af4242deb300d852e13e429b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.034755 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8452d964af4242deb300d852e13e429b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:16.036693 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:16.087548 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8452d964af4242deb300d852e13e429b', 'request_id': 'req-878ccced-0fc4-475f-95bb-6a4dde6d37d5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:16.097097 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8452d964af4242deb300d852e13e429b', 'request_id': 'req-878ccced-0fc4-475f-95bb-6a4dde6d37d5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:16.103426 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8452d964af4242deb300d852e13e429b', 'request_id': 'req-878ccced-0fc4-475f-95bb-6a4dde6d37d5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:16.113850 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] The token cache is being invalidate because project 8452d964af4242deb300d852e13e429b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 19:03:16.114133 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:16.114796 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-878ccced-0fc4-475f-95bb-6a4dde6d37d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:16.116375 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1440/2894] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 19:03:15 2022] DELETE /identity/v3/projects/8452d964af4242deb300d852e13e429b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:16.121712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f09b1-1ab6-4fdc-bb40-9ef4b9221d2c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.122259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f09b1-1ab6-4fdc-bb40-9ef4b9221d2c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.122259 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f09b1-1ab6-4fdc-bb40-9ef4b9221d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.141483 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-169f09b1-1ab6-4fdc-bb40-9ef4b9221d2c None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:16.142712 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-169f09b1-1ab6-4fdc-bb40-9ef4b9221d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:16.211480 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1455/2895] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:16.223134 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba677-33b7-4dd5-bc02-f13807153f06 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.223417 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba677-33b7-4dd5-bc02-f13807153f06 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.223688 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba677-33b7-4dd5-bc02-f13807153f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.243543 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-338ba677-33b7-4dd5-bc02-f13807153f06 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:16.244779 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-338ba677-33b7-4dd5-bc02-f13807153f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:16.253841 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1441/2896] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:16.258271 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f09b1-1ab6-4fdc-bb40-9ef4b9221d2c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:16.268981 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f09b1-1ab6-4fdc-bb40-9ef4b9221d2c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:16.275274 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c87dd5-f7fa-4f7c-9989-8ccebb076e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:16.275930 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c87dd5-f7fa-4f7c-9989-8ccebb076e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.276209 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c87dd5-f7fa-4f7c-9989-8ccebb076e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.276478 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c87dd5-f7fa-4f7c-9989-8ccebb076e65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.277266 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c87dd5-f7fa-4f7c-9989-8ccebb076e65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Apr 29 19:03:16.277569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c87dd5-f7fa-4f7c-9989-8ccebb076e65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:16.279172 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c87dd5-f7fa-4f7c-9989-8ccebb076e65 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:16.291854 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1456/2897] 10.222.0.50 () {58 vars in 1152 bytes} [Fri Apr 29 19:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Apr 29 19:03:16.295565 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba677-33b7-4dd5-bc02-f13807153f06 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:16.305474 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba677-33b7-4dd5-bc02-f13807153f06 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:16.311451 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340f9bd-a0cc-4c65-8de3-79ef3268ccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:16.312100 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340f9bd-a0cc-4c65-8de3-79ef3268ccfe admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.312379 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340f9bd-a0cc-4c65-8de3-79ef3268ccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.312695 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340f9bd-a0cc-4c65-8de3-79ef3268ccfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.313386 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340f9bd-a0cc-4c65-8de3-79ef3268ccfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:16.315431 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340f9bd-a0cc-4c65-8de3-79ef3268ccfe admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:16.342741 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1442/2898] 10.222.0.50 () {60 vars in 1152 bytes} [Fri Apr 29 19:03:16 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Apr 29 19:03:16.347102 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c87dd5-f7fa-4f7c-9989-8ccebb076e65 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:16.357463 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c87dd5-f7fa-4f7c-9989-8ccebb076e65 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:16.412104 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4e7f2-177c-4713-97d1-ef2995ff9fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:16.412792 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4e7f2-177c-4713-97d1-ef2995ff9fdc admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.413072 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4e7f2-177c-4713-97d1-ef2995ff9fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.413342 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4e7f2-177c-4713-97d1-ef2995ff9fdc admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.414039 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4e7f2-177c-4713-97d1-ef2995ff9fdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:16.415936 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4e7f2-177c-4713-97d1-ef2995ff9fdc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:16.440602 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1457/2899] 10.222.0.50 () {60 vars in 1143 bytes} [Fri Apr 29 19:03:16 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Apr 29 19:03:16.444808 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340f9bd-a0cc-4c65-8de3-79ef3268ccfe admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:16.455473 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340f9bd-a0cc-4c65-8de3-79ef3268ccfe admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:16.460983 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350708a-7979-4e21-91a9-101355cddb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:16.461646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350708a-7979-4e21-91a9-101355cddb3d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.461928 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350708a-7979-4e21-91a9-101355cddb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.462190 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350708a-7979-4e21-91a9-101355cddb3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.462902 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350708a-7979-4e21-91a9-101355cddb3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:16.464511 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350708a-7979-4e21-91a9-101355cddb3d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:16.475715 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1443/2900] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:03:16.481309 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4e7f2-177c-4713-97d1-ef2995ff9fdc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:16.493276 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4e7f2-177c-4713-97d1-ef2995ff9fdc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:16.497841 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac2fcb-0f9b-4f07-9ff5-c72aa1556a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:16.498498 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac2fcb-0f9b-4f07-9ff5-c72aa1556a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.498778 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac2fcb-0f9b-4f07-9ff5-c72aa1556a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.499053 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac2fcb-0f9b-4f07-9ff5-c72aa1556a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.499786 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac2fcb-0f9b-4f07-9ff5-c72aa1556a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:16.501479 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac2fcb-0f9b-4f07-9ff5-c72aa1556a91 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:16.511265 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1458/2901] 10.222.0.50 () {58 vars in 1121 bytes} [Fri Apr 29 19:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Apr 29 19:03:16.515574 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350708a-7979-4e21-91a9-101355cddb3d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:16.525412 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350708a-7979-4e21-91a9-101355cddb3d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:16.530362 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca743eb-50fb-4b93-805f-16342d88701c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:16.531238 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca743eb-50fb-4b93-805f-16342d88701c admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.531514 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca743eb-50fb-4b93-805f-16342d88701c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.531781 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca743eb-50fb-4b93-805f-16342d88701c admin admin] PATH_INFO: `/v3/projects/98797cf44a3d476f9e77f25901007398/users/06d2aac7dfd8428cb5ac1934e414002a/roles/3385b41b990e4c6b88d77260a7822c7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.546337 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca743eb-50fb-4b93-805f-16342d88701c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98797cf44a3d476f9e77f25901007398, user_id=06d2aac7dfd8428cb5ac1934e414002a, role_id=3385b41b990e4c6b88d77260a7822c7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:16.549034 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca743eb-50fb-4b93-805f-16342d88701c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:16.563765 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1444/2902] 10.222.0.50 () {60 vars in 1482 bytes} [Fri Apr 29 19:03:16 2022] PUT /identity/v3/projects/98797cf44a3d476f9e77f25901007398/users/06d2aac7dfd8428cb5ac1934e414002a/roles/3385b41b990e4c6b88d77260a7822c7e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 19:03:16.576326 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afad062-dcd6-4416-9f3f-c8d8d621f630 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.576716 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afad062-dcd6-4416-9f3f-c8d8d621f630 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.577100 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afad062-dcd6-4416-9f3f-c8d8d621f630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.608216 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0afad062-dcd6-4416-9f3f-c8d8d621f630 None None] MFA Rules not processed for user `06d2aac7dfd8428cb5ac1934e414002a`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:16.609452 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0afad062-dcd6-4416-9f3f-c8d8d621f630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:16.678050 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1459/2903] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:16.689598 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca743eb-50fb-4b93-805f-16342d88701c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:16.691645 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3ca743eb-50fb-4b93-805f-16342d88701c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:16.751330 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca743eb-50fb-4b93-805f-16342d88701c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:16.755899 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c322bb-1f4d-4469-a271-2323a56be7c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4fe640bb564a8abe872cb1ec74dc7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:16.756526 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c322bb-1f4d-4469-a271-2323a56be7c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:16.756844 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c322bb-1f4d-4469-a271-2323a56be7c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:16.757112 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c322bb-1f4d-4469-a271-2323a56be7c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:16.765791 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c322bb-1f4d-4469-a271-2323a56be7c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:16.767714 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c322bb-1f4d-4469-a271-2323a56be7c0 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:16.780068 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1445/2904] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:03:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Apr 29 19:03:38.465565 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afad062-dcd6-4416-9f3f-c8d8d621f630 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:38.538754 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afad062-dcd6-4416-9f3f-c8d8d621f630 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:38.545160 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:38.546040 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:38.546315 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:38.546582 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] PATH_INFO: `/v3/users/06d2aac7dfd8428cb5ac1934e414002a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:38.547960 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d2aac7dfd8428cb5ac1934e414002a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:38.549985 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:38.603470 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d2aac7dfd8428cb5ac1934e414002a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:38.604153 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d2aac7dfd8428cb5ac1934e414002a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:38.616306 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d2aac7dfd8428cb5ac1934e414002a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:38.626429 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d2aac7dfd8428cb5ac1934e414002a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:38.634578 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d2aac7dfd8428cb5ac1934e414002a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:38.640766 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1460/2905] 10.222.0.50 () {58 vars in 1223 bytes} [Fri Apr 29 19:03:38 2022] DELETE /identity/v3/users/06d2aac7dfd8428cb5ac1934e414002a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Apr 29 19:03:38.645446 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6323bf-3fc9-4891-a91b-995824a36110 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:38.645960 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6323bf-3fc9-4891-a91b-995824a36110 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:38.646073 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6323bf-3fc9-4891-a91b-995824a36110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:38.663962 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6e6323bf-3fc9-4891-a91b-995824a36110 None None] MFA Rules not processed for user `a74b9eac8d87492aad683f6da38fa8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Apr 29 19:03:38.665333 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6e6323bf-3fc9-4891-a91b-995824a36110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:38.731729 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1446/2906] 10.222.0.50 () {58 vars in 957 bytes} [Fri Apr 29 19:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Apr 29 19:03:38.741673 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:38.743166 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:38.799373 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c201115-78ec-48f4-aa89-71e3902c311b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:38.803569 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5629f-5166-4d5d-a90c-f24f98c95622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07849b2925334763b119111ed52dd34c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:38.804258 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5629f-5166-4d5d-a90c-f24f98c95622 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:38.804564 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5629f-5166-4d5d-a90c-f24f98c95622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:38.804848 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5629f-5166-4d5d-a90c-f24f98c95622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:38.814852 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5629f-5166-4d5d-a90c-f24f98c95622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:38.816293 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5629f-5166-4d5d-a90c-f24f98c95622 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:38.825976 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1461/2907] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Apr 29 19:03:39.092804 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6323bf-3fc9-4891-a91b-995824a36110 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:03:39.094489 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6e6323bf-3fc9-4891-a91b-995824a36110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:03:39.106376 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6323bf-3fc9-4891-a91b-995824a36110 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:03:39.111103 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a74b9eac8d87492aad683f6da38fa8b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28300f353c4c4c27a229238b2a238c8e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:03:39.111970 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:03:39.112250 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:03:39.112555 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] PATH_INFO: `/v3/projects/98797cf44a3d476f9e77f25901007398` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:03:39.113763 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98797cf44a3d476f9e77f25901007398)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:03:39.115673 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:03:39.163380 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98797cf44a3d476f9e77f25901007398', 'request_id': 'req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:39.171452 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98797cf44a3d476f9e77f25901007398', 'request_id': 'req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:39.179527 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98797cf44a3d476f9e77f25901007398', 'request_id': 'req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:39.193215 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] The token cache is being invalidate because project 98797cf44a3d476f9e77f25901007398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Apr 29 19:03:39.193503 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:39.194169 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f04af4e8-8a8c-496b-98a8-e7bc381d4de0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Apr 29 19:03:39.196646 dv-cinder-839987-1-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1447/2908] 10.222.0.50 () {58 vars in 1232 bytes} [Fri Apr 29 19:03:39 2022] DELETE /identity/v3/projects/98797cf44a3d476f9e77f25901007398 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Apr 29 19:07:43.600677 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5629f-5166-4d5d-a90c-f24f98c95622 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Apr 29 19:07:43.602434 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-54d5629f-5166-4d5d-a90c-f24f98c95622 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:07:43.752541 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5629f-5166-4d5d-a90c-f24f98c95622 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Apr 29 19:07:43.816057 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab56e0-f989-46da-959f-bf2c9891f0d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e934c8e7dd0425cac68eb91e5e2c0ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7398b586ccc4806bdaa4365ce558959', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Apr 29 19:07:43.816900 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab56e0-f989-46da-959f-bf2c9891f0d2 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Apr 29 19:07:43.817208 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab56e0-f989-46da-959f-bf2c9891f0d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Apr 29 19:07:43.817482 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab56e0-f989-46da-959f-bf2c9891f0d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Apr 29 19:07:43.819382 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a9ab56e0-f989-46da-959f-bf2c9891f0d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Apr 29 19:07:43.831554 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab56e0-f989-46da-959f-bf2c9891f0d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Apr 29 19:07:43.833253 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab56e0-f989-46da-959f-bf2c9891f0d2 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Apr 29 19:07:43.843386 dv-cinder-839987-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1462/2909] 10.222.0.50 () {60 vars in 1365 bytes} [Fri Apr 29 19:07:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)